[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:20 dist-test-slave-jztv krb5kdc[7837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:20 dist-test-slave-jztv krb5kdc[7837](info): set up 2 sockets
Aug 08 18:27:20 dist-test-slave-jztv krb5kdc[7837](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 18:27:21.417917 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:27:21 dist-test-slave-jztv krb5kdc[7837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:27:21.553431 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:41943
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:36927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:41943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:21.684784 7858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:21.685135 7858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:21.685225 7858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:21.690100 7858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:21.690194 7858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:21.690254 7858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:21.690294 7858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:21.690327 7858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:21.696257 7858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:36927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:41943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:41943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:21.697683 7858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:21.698925 7858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:21.708660 7865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:21.708729 7858 server_base.cc:1034] running on GCE node
W20260808 18:27:21.708711 7864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:21.708671 7867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:21.709549 7858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:21.711167 7858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:21.712364 7858 hybrid_clock.cc:648] HybridClock initialized: now 1786213641712334 us; error 76 us; skew 500 ppm
Aug 08 18:27:21 dist-test-slave-jztv krb5kdc[7837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213641, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:21.715377 7858 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:27:21.716596 7858 webserver.cc:460] Webserver started at http://127.7.164.126:43111/ using document root <none> and password file <none>
I20260808 18:27:21.717243 7858 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:21.717341 7858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:21.717697 7858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:21.719978 7858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "b1d31c70fd534969a3591246e4b999ba"
format_stamp: "Formatted at 2026-08-08 18:27:21 on dist-test-slave-jztv"
server_key: "0ce959a8047ac31ced182308a341cf04"
server_key_iv: "537881087a51690e064e96a9f5b33478"
server_key_version: "encryptionkey@0"
I20260808 18:27:21.720582 7858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "b1d31c70fd534969a3591246e4b999ba"
format_stamp: "Formatted at 2026-08-08 18:27:21 on dist-test-slave-jztv"
server_key: "0ce959a8047ac31ced182308a341cf04"
server_key_iv: "537881087a51690e064e96a9f5b33478"
server_key_version: "encryptionkey@0"
I20260808 18:27:21.724716 7858 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:27:21.727404 7885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:21.728709 7858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:27:21.728874 7858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "b1d31c70fd534969a3591246e4b999ba"
format_stamp: "Formatted at 2026-08-08 18:27:21 on dist-test-slave-jztv"
server_key: "0ce959a8047ac31ced182308a341cf04"
server_key_iv: "537881087a51690e064e96a9f5b33478"
server_key_version: "encryptionkey@0"
I20260808 18:27:21.728999 7858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:21.755188 7858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:21.759047 7858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:21.759269 7858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:21.767652 7858 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:41943
I20260808 18:27:21.767673 7957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:41943 every 8 connection(s)
I20260808 18:27:21.768762 7858 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:21.771780 7958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:21.774134 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 7858
I20260808 18:27:21.774326 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:21.776022 7825 external_mini_cluster.cc:1428] Setting key 26c373822e50e936c7320922896be52e
I20260808 18:27:21.779210 7958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba: Bootstrap starting.
I20260808 18:27:21.781695 7958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:21.782528 7958 log.cc:824] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:21.784708 7958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba: No bootstrap required, opened a new log
I20260808 18:27:21.788928 7958 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d31c70fd534969a3591246e4b999ba" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41943 } }
I20260808 18:27:21.789191 7958 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:21.789237 7958 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d31c70fd534969a3591246e4b999ba, State: Initialized, Role: FOLLOWER
I20260808 18:27:21.789711 7958 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d31c70fd534969a3591246e4b999ba" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41943 } }
I20260808 18:27:21.789844 7958 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:21.789930 7958 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:21.790048 7958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:21.791013 7958 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d31c70fd534969a3591246e4b999ba" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41943 } }
I20260808 18:27:21.791365 7958 leader_election.cc:302] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1d31c70fd534969a3591246e4b999ba; no voters:
I20260808 18:27:21.791721 7958 leader_election.cc:288] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:21.791865 7964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:21.792157 7964 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [term 1 LEADER]: Becoming Leader. State: Replica: b1d31c70fd534969a3591246e4b999ba, State: Running, Role: LEADER
I20260808 18:27:21.792526 7964 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d31c70fd534969a3591246e4b999ba" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41943 } }
I20260808 18:27:21.792932 7958 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:21.794660 7965 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1d31c70fd534969a3591246e4b999ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d31c70fd534969a3591246e4b999ba" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41943 } } }
I20260808 18:27:21.794819 7965 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:21.795296 7966 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1d31c70fd534969a3591246e4b999ba. Latest consensus state: current_term: 1 leader_uuid: "b1d31c70fd534969a3591246e4b999ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d31c70fd534969a3591246e4b999ba" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41943 } } }
I20260808 18:27:21.796029 7966 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba [sys.catalog]: This master's current role is: LEADER
Aug 08 18:27:21 dist-test-slave-jztv krb5kdc[7837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
W20260808 18:27:21.798501 7983 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:27:21.798596 7983 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:27:21.798709 7981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:21.799592 7981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:21.804137 7960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:21.780069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54416 (local address 127.7.164.126:41943)
0808 18:27:21.780595 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:21.780620 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:21.781685 (+ 1065us) server_negotiation.cc:407] Connection header received
0808 18:27:21.785227 (+ 3542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:21.785249 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:21.785630 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:21.785986 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:21.788087 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:21.789539 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:21.790971 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:21.791682 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:21.791842 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:21.797612 (+ 5770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:21.797619 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:21.797634 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:21.797668 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:21.799840 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:21.800440 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:21.800445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:21.800449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:21.800565 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:21.800968 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:21.800971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:21.800972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:21.801569 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:21.801777 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:21.802514 (+ 737us) server_negotiation.cc:299] Negotiation successful
0808 18:27:21.802810 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":157,"threads_started":1}
I20260808 18:27:21.805933 7981 catalog_manager.cc:1347] Generated new cluster ID: add37a021b264d81868d43e6a05cedb1
I20260808 18:27:21.805997 7981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:21.819883 7981 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:21.820914 7981 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:21.825423 7981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1d31c70fd534969a3591246e4b999ba: Generated new TSK 0
I20260808 18:27:21.826156 7981 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:27:21.861819 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41943
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:36927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:21.983296 7990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:21.983592 7990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:21.983667 7990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:21.987820 7990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:21.987983 7990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:21.988127 7990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:27:21.994719 7990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:36927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41943
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:21.995801 7990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:21.996812 7990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:22.006343 7997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:22.006345 8002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:22.006563 7998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:22.006971 7990 server_base.cc:1034] running on GCE node
I20260808 18:27:22.007347 7990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:22.007978 7990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:22.009166 7990 hybrid_clock.cc:648] HybridClock initialized: now 1786213642009136 us; error 42 us; skew 500 ppm
Aug 08 18:27:22 dist-test-slave-jztv krb5kdc[7837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213642, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:22.011919 7990 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:27:22.012802 7990 webserver.cc:460] Webserver started at http://127.7.164.65:45461/ using document root <none> and password file <none>
I20260808 18:27:22.013351 7990 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:22.013443 7990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:22.013690 7990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:22.015760 7990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "22e55af6d39545659796a035e0631187"
format_stamp: "Formatted at 2026-08-08 18:27:22 on dist-test-slave-jztv"
server_key: "e2174b732fdd104c59b380b7b7b46831"
server_key_iv: "48ec981037bbbff1bc5033ae91367387"
server_key_version: "encryptionkey@0"
I20260808 18:27:22.016335 7990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "22e55af6d39545659796a035e0631187"
format_stamp: "Formatted at 2026-08-08 18:27:22 on dist-test-slave-jztv"
server_key: "e2174b732fdd104c59b380b7b7b46831"
server_key_iv: "48ec981037bbbff1bc5033ae91367387"
server_key_version: "encryptionkey@0"
I20260808 18:27:22.020083 7990 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 18:27:22.022516 8011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:22.023697 7990 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:27:22.023838 7990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "22e55af6d39545659796a035e0631187"
format_stamp: "Formatted at 2026-08-08 18:27:22 on dist-test-slave-jztv"
server_key: "e2174b732fdd104c59b380b7b7b46831"
server_key_iv: "48ec981037bbbff1bc5033ae91367387"
server_key_version: "encryptionkey@0"
I20260808 18:27:22.023954 7990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:22.047958 7990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:22.050900 7990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:22.051116 7990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:22.051733 7990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:22.052587 7990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:22.052659 7990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:22.052731 7990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:22.052781 7990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:22.064275 7990 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:45925
I20260808 18:27:22.064301 8165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:45925 every 8 connection(s)
I20260808 18:27:22.065229 7990 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:27:22.070637 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 7990
I20260808 18:27:22.070746 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:27:22.071051 7825 external_mini_cluster.cc:1428] Setting key c83d615905f73a667399aa9d9d9e421b
Aug 08 18:27:22 dist-test-slave-jztv krb5kdc[7837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213642, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:22.080200 7960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:22.067047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:58973 (local address 127.7.164.126:41943)
0808 18:27:22.067228 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:22.067234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:22.068043 (+ 809us) server_negotiation.cc:407] Connection header received
0808 18:27:22.069565 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:22.069568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:22.069616 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:22.069690 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:22.071137 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.071778 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.073404 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.073635 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.073715 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:22.076491 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:22.076499 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:22.076502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:22.076526 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:22.078033 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:22.078610 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:22.078613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:22.078615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:22.078703 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:22.079066 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:22.079069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:22.079071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:22.079261 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:22.079383 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:22.079883 (+ 500us) server_negotiation.cc:299] Negotiation successful
0808 18:27:22.080016 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 18:27:22.080968 8168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:22.067427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41943 (local address 127.7.164.65:58973)
0808 18:27:22.067883 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:22.067926 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:22.069363 (+ 1437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:22.069857 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:22.069863 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:22.070257 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:22.070964 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:22.070986 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.071932 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.071935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:22.073274 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:22.073289 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.073791 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.073797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:22.074078 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:22.074864 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:22.074885 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:22.076311 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:22.078167 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:22.078173 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:22.078188 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:22.078500 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:22.078810 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:22.078814 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:22.078815 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:22.078958 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:22.079395 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:22.079403 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:22.079641 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 18:27:22.079824 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 18:27:22.080155 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":120,"threads_started":1}
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:27:22.082285 8166 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41943
I20260808 18:27:22.082526 8166 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:22.082996 8166 heartbeater.cc:511] Master 127.7.164.126:41943 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
I20260808 18:27:22.084754 7906 ts_manager.cc:194] Registered new tserver with Master: 22e55af6d39545659796a035e0631187 (127.7.164.65:45925)
I20260808 18:27:22.086009 7906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:58973
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:27:22.100845 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41943
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:36927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:22.217235 8178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:22.217511 8178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:22.217557 8178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:22.221670 8178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:22.221740 8178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:22.221801 8178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:27:22.227281 8178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:36927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41943
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:22.228506 8178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:22.229573 8178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:22.239724 8184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:22.239724 8185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:22.239728 8192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:22.240005 8178 server_base.cc:1034] running on GCE node
I20260808 18:27:22.240629 8178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:22.241358 8178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:22.242560 8178 hybrid_clock.cc:648] HybridClock initialized: now 1786213642242542 us; error 29 us; skew 500 ppm
Aug 08 18:27:22 dist-test-slave-jztv krb5kdc[7837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213642, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:22.245424 8178 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:27:22.246562 8178 webserver.cc:460] Webserver started at http://127.7.164.66:36205/ using document root <none> and password file <none>
I20260808 18:27:22.247169 8178 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:22.247280 8178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:22.247489 8178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:22.249629 8178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "6645c9ce696a4b4f9ce72da25e47bca9"
format_stamp: "Formatted at 2026-08-08 18:27:22 on dist-test-slave-jztv"
server_key: "c5a572514c4005c14ab8596fdd08ef85"
server_key_iv: "99b1a084072094234bb60588b429aa90"
server_key_version: "encryptionkey@0"
I20260808 18:27:22.250168 8178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "6645c9ce696a4b4f9ce72da25e47bca9"
format_stamp: "Formatted at 2026-08-08 18:27:22 on dist-test-slave-jztv"
server_key: "c5a572514c4005c14ab8596fdd08ef85"
server_key_iv: "99b1a084072094234bb60588b429aa90"
server_key_version: "encryptionkey@0"
I20260808 18:27:22.254099 8178 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 18:27:22.256851 8206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:22.258668 8178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:27:22.258826 8178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "6645c9ce696a4b4f9ce72da25e47bca9"
format_stamp: "Formatted at 2026-08-08 18:27:22 on dist-test-slave-jztv"
server_key: "c5a572514c4005c14ab8596fdd08ef85"
server_key_iv: "99b1a084072094234bb60588b429aa90"
server_key_version: "encryptionkey@0"
I20260808 18:27:22.258935 8178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:22.285634 8178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:22.288705 8178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:22.288875 8178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:22.289433 8178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:22.290344 8178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:22.290401 8178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:22.290445 8178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:22.290462 8178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:22.300282 8178 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:38363
I20260808 18:27:22.300367 8359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:38363 every 8 connection(s)
I20260808 18:27:22.301234 8178 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:27:22.308384 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 8178
I20260808 18:27:22.308530 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:27:22.308780 7825 external_mini_cluster.cc:1428] Setting key ef8f587b666a2feb60927345f722c5af
Aug 08 18:27:22 dist-test-slave-jztv krb5kdc[7837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213642, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:22.315851 7960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:22.303251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:52267 (local address 127.7.164.126:41943)
0808 18:27:22.303395 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:22.303398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:22.304356 (+ 958us) server_negotiation.cc:407] Connection header received
0808 18:27:22.305742 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:22.305745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:22.305793 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:22.305876 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:22.307139 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.307686 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.308900 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.309150 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.309265 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:22.312116 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:22.312122 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:22.312128 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:22.312165 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:22.313670 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:22.314201 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:22.314205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:22.314209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:22.314341 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:22.314701 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:22.314706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:22.314708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:22.314899 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:22.315007 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:22.315542 (+ 535us) server_negotiation.cc:299] Negotiation successful
0808 18:27:22.315675 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:27:22.316854 8366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:22.303643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41943 (local address 127.7.164.66:52267)
0808 18:27:22.304151 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:22.304214 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:22.305538 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:22.306046 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:22.306053 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:22.306518 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:22.307004 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:22.307018 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.307797 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.307800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:22.308779 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:22.308789 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.309312 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.309337 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:22.309563 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:22.310262 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:22.310290 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:22.311938 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:22.313777 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:22.313781 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:22.313794 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:22.314080 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:22.314439 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:22.314443 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:22.314445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:22.314596 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:22.315029 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:22.315038 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:22.315284 (+ 246us) client_negotiation.cc:769] Sending connection context
0808 18:27:22.315482 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 18:27:22.315813 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":114,"threads_started":1}
I20260808 18:27:22.318177 8363 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41943
I20260808 18:27:22.318508 8363 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:22.319031 8363 heartbeater.cc:511] Master 127.7.164.126:41943 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
I20260808 18:27:22.320163 7906 ts_manager.cc:194] Registered new tserver with Master: 6645c9ce696a4b4f9ce72da25e47bca9 (127.7.164.66:38363)
I20260808 18:27:22.320833 7906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:52267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:27:22.340888 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41943
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:36927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:22.453938 8371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:22.454172 8371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:22.454267 8371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:22.458287 8371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:22.458336 8371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:22.458400 8371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:27:22.463624 8371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:36927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41943
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:22.464583 8371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:22.465530 8371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:22.474947 8380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:22.474947 8381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:22.475085 8371 server_base.cc:1034] running on GCE node
W20260808 18:27:22.474975 8383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:22.475611 8371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:22.476276 8371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:22.477458 8371 hybrid_clock.cc:648] HybridClock initialized: now 1786213642477442 us; error 27 us; skew 500 ppm
Aug 08 18:27:22 dist-test-slave-jztv krb5kdc[7837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213642, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:22.480175 8371 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:27:22.481053 8371 webserver.cc:460] Webserver started at http://127.7.164.67:33069/ using document root <none> and password file <none>
I20260808 18:27:22.481602 8371 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:22.481662 8371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:22.481905 8371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:22.483951 8371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "e2b011fce4044fa8a6e1307e7141a37a"
format_stamp: "Formatted at 2026-08-08 18:27:22 on dist-test-slave-jztv"
server_key: "08e39c39dd98dd390c0733f9b314d9e3"
server_key_iv: "ee7a4aef6ddee0b0057137ebc7b0f997"
server_key_version: "encryptionkey@0"
I20260808 18:27:22.484500 8371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "e2b011fce4044fa8a6e1307e7141a37a"
format_stamp: "Formatted at 2026-08-08 18:27:22 on dist-test-slave-jztv"
server_key: "08e39c39dd98dd390c0733f9b314d9e3"
server_key_iv: "ee7a4aef6ddee0b0057137ebc7b0f997"
server_key_version: "encryptionkey@0"
I20260808 18:27:22.488248 8371 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:27:22.490595 8399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:22.491704 8371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260808 18:27:22.491824 8371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "e2b011fce4044fa8a6e1307e7141a37a"
format_stamp: "Formatted at 2026-08-08 18:27:22 on dist-test-slave-jztv"
server_key: "08e39c39dd98dd390c0733f9b314d9e3"
server_key_iv: "ee7a4aef6ddee0b0057137ebc7b0f997"
server_key_version: "encryptionkey@0"
I20260808 18:27:22.491933 8371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:22.505911 8371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:22.508658 8371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:22.508857 8371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:22.509413 8371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:22.510201 8371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:22.510313 8371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:22.510376 8371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:22.510416 8371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:22.519771 8371 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:34609
I20260808 18:27:22.519793 8564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:34609 every 8 connection(s)
I20260808 18:27:22.520671 8371 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:27:22.527940 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 8371
I20260808 18:27:22.528070 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:27:22.528355 7825 external_mini_cluster.cc:1428] Setting key 22c9b613f7b2f713262d19d3993ef3c9
Aug 08 18:27:22 dist-test-slave-jztv krb5kdc[7837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213642, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:22.535446 7960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:22.522611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:51855 (local address 127.7.164.126:41943)
0808 18:27:22.522761 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:22.522765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:22.523608 (+ 843us) server_negotiation.cc:407] Connection header received
0808 18:27:22.525005 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:22.525009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:22.525068 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:22.525176 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:22.526533 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.527081 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.528682 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.528907 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.528991 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:22.531940 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:22.531943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:22.531945 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:22.531972 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:22.533324 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:22.533827 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:22.533830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:22.533831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:22.533965 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:22.534376 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:22.534383 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:22.534385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:22.534637 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:22.534721 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:22.535214 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 18:27:22.535323 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 18:27:22.536267 8575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:22.522988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41943 (local address 127.7.164.67:51855)
0808 18:27:22.523453 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:22.523494 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:22.524785 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:22.525384 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:22.525391 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:22.525757 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:22.526382 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:22.526399 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.527216 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.527220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:22.528524 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:22.528536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.529055 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.529060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:22.529230 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:22.529926 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:22.529943 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:22.531767 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:22.533431 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:22.533437 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:22.533450 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:22.533730 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:22.534054 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:22.534057 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:22.534058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:22.534197 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:22.534736 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:22.534741 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:22.534991 (+ 250us) client_negotiation.cc:769] Sending connection context
0808 18:27:22.535148 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 18:27:22.535414 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":123,"threads_started":1}
I20260808 18:27:22.537477 8567 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41943
I20260808 18:27:22.537739 8567 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:22.538316 8567 heartbeater.cc:511] Master 127.7.164.126:41943 requested a full tablet report, sending...
I20260808 18:27:22.539383 7906 ts_manager.cc:194] Registered new tserver with Master: e2b011fce4044fa8a6e1307e7141a37a (127.7.164.67:34609)
I20260808 18:27:22.539932 7906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:51855
I20260808 18:27:22.543074 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 18:27:22 dist-test-slave-jztv krb5kdc[7837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213642, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:27:22 dist-test-slave-jztv krb5kdc[7837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213642, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.164.65@KRBTEST.COM
I20260808 18:27:22.570849 8581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:22.560962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54258 (local address 127.7.164.65:45925)
0808 18:27:22.561296 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:22.561312 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:22.561341 (+ 29us) server_negotiation.cc:407] Connection header received
0808 18:27:22.561461 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:22.561468 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:22.561660 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:22.561825 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:22.562661 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.563458 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.564321 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.564699 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.564816 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:22.567328 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:22.567338 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:22.567352 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:22.567377 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:22.568919 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:22.569393 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:22.569399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:22.569400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:22.569483 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:22.569767 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:22.569771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:22.569773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:22.570033 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:22.570291 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:22.570469 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 18:27:22.570662 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":93,"threads_started":1}
W20260808 18:27:22.571857 8055 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54258
Aug 08 18:27:22 dist-test-slave-jztv krb5kdc[7837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:27:22 dist-test-slave-jztv krb5kdc[7837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.65@KRBTEST.COM
I20260808 18:27:22.607409 8581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:22.595784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54260 (local address 127.7.164.65:45925)
0808 18:27:22.595932 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:22.595936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:22.598549 (+ 2613us) server_negotiation.cc:407] Connection header received
0808 18:27:22.598725 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:22.598729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:22.598789 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:22.598883 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:22.599638 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.600192 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.601050 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:22.601296 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:22.601409 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:22.604201 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:22.604204 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:22.604206 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:22.604233 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:22.605691 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:22.606111 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:22.606115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:22.606117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:22.606203 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:22.606546 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:22.606550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:22.606551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:22.606727 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:22.606867 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:22.607116 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 18:27:22.607241 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:27:22.608837 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 7990
I20260808 18:27:22.623823 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 8178
I20260808 18:27:22.629873 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 8371
I20260808 18:27:22.647461 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 7858
2026-08-08T18:27:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2491 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:22 dist-test-slave-jztv krb5kdc[8592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:22 dist-test-slave-jztv krb5kdc[8592](info): set up 2 sockets
Aug 08 18:27:22 dist-test-slave-jztv krb5kdc[8592](info): commencing operation
krb5kdc: starting...
W20260808 18:27:23.868541 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:27:23 dist-test-slave-jztv krb5kdc[8592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:27:23.997715 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:41041
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:41041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:24.107460 8607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:24.107723 8607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:24.107764 8607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:24.111730 8607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:24.111783 8607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:24.111800 8607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:24.111817 8607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:24.111881 8607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:24.117131 8607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:41041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:41041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:24.118170 8607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:24.119211 8607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:24.127600 8617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:24.127609 8618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:24.127797 8620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:24.127928 8607 server_base.cc:1034] running on GCE node
I20260808 18:27:24.128511 8607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:24.129565 8607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:24.130728 8607 hybrid_clock.cc:648] HybridClock initialized: now 1786213644130708 us; error 29 us; skew 500 ppm
Aug 08 18:27:24 dist-test-slave-jztv krb5kdc[8592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213644, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:24.133456 8607 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:27:24.134500 8607 webserver.cc:460] Webserver started at http://127.7.164.126:43727/ using document root <none> and password file <none>
I20260808 18:27:24.135070 8607 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:24.135130 8607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:24.135381 8607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:24.137420 8607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "f8f7c3fab12d4543b319478b51372c86"
format_stamp: "Formatted at 2026-08-08 18:27:24 on dist-test-slave-jztv"
server_key: "0f44065e6c2131089f154576d5c7aa62"
server_key_iv: "ebd66dde4d08d95a3025348d6baa2ee7"
server_key_version: "encryptionkey@0"
I20260808 18:27:24.138005 8607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "f8f7c3fab12d4543b319478b51372c86"
format_stamp: "Formatted at 2026-08-08 18:27:24 on dist-test-slave-jztv"
server_key: "0f44065e6c2131089f154576d5c7aa62"
server_key_iv: "ebd66dde4d08d95a3025348d6baa2ee7"
server_key_version: "encryptionkey@0"
I20260808 18:27:24.141759 8607 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:27:24.144233 8631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:24.145457 8607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:24.145603 8607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "f8f7c3fab12d4543b319478b51372c86"
format_stamp: "Formatted at 2026-08-08 18:27:24 on dist-test-slave-jztv"
server_key: "0f44065e6c2131089f154576d5c7aa62"
server_key_iv: "ebd66dde4d08d95a3025348d6baa2ee7"
server_key_version: "encryptionkey@0"
I20260808 18:27:24.145710 8607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:24.159183 8607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:24.162202 8607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:24.162446 8607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:24.169761 8607 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:41041
I20260808 18:27:24.169786 8712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:41041 every 8 connection(s)
I20260808 18:27:24.170747 8607 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:24.173484 8713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:24.174453 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 8607
I20260808 18:27:24.174546 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:24.174835 7825 external_mini_cluster.cc:1428] Setting key 256e2c74460b1b22b53f6f5cffed8048
I20260808 18:27:24.180261 8713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86: Bootstrap starting.
I20260808 18:27:24.182843 8713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:24.183574 8713 log.cc:824] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:27:24 dist-test-slave-jztv krb5kdc[8592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:24.185894 8713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86: No bootstrap required, opened a new log
I20260808 18:27:24.190485 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:24.176285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50582 (local address 127.7.164.126:41041)
0808 18:27:24.176713 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:24.176741 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:24.176785 (+ 44us) server_negotiation.cc:407] Connection header received
0808 18:27:24.177642 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:24.177665 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:24.178070 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:24.178476 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:24.179127 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.180589 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.181402 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.182076 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.182284 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:24.184969 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:24.184976 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:24.184992 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:24.185026 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:24.187080 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:24.187508 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:24.187514 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:24.187520 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:24.187665 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:24.187930 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:24.187934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:24.187935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:24.188617 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:24.188806 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:24.189177 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 18:27:24.189495 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":113,"threads_started":1}
I20260808 18:27:24.190879 8713 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8f7c3fab12d4543b319478b51372c86" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41041 } }
I20260808 18:27:24.191102 8713 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:24.191174 8713 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8f7c3fab12d4543b319478b51372c86, State: Initialized, Role: FOLLOWER
I20260808 18:27:24.191646 8713 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8f7c3fab12d4543b319478b51372c86" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41041 } }
I20260808 18:27:24.191803 8713 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:24.191892 8713 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:24.192004 8713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:24.193091 8713 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8f7c3fab12d4543b319478b51372c86" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41041 } }
I20260808 18:27:24.193428 8713 leader_election.cc:302] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8f7c3fab12d4543b319478b51372c86; no voters:
I20260808 18:27:24.193734 8713 leader_election.cc:288] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:24.193861 8722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:24.194118 8722 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [term 1 LEADER]: Becoming Leader. State: Replica: f8f7c3fab12d4543b319478b51372c86, State: Running, Role: LEADER
I20260808 18:27:24.194506 8722 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8f7c3fab12d4543b319478b51372c86" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41041 } }
I20260808 18:27:24.194986 8713 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:24.196293 8724 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8f7c3fab12d4543b319478b51372c86. Latest consensus state: current_term: 1 leader_uuid: "f8f7c3fab12d4543b319478b51372c86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8f7c3fab12d4543b319478b51372c86" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41041 } } }
I20260808 18:27:24.196293 8723 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8f7c3fab12d4543b319478b51372c86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8f7c3fab12d4543b319478b51372c86" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41041 } } }
I20260808 18:27:24.196457 8724 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:24.196457 8723 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:24.196879 8734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:24.200052 8734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:24.205307 8734 catalog_manager.cc:1347] Generated new cluster ID: 4c477b9130c840628daef30a8eb2ad70
I20260808 18:27:24.205447 8734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:24.216372 8734 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:24.217231 8734 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:24.227190 8734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8f7c3fab12d4543b319478b51372c86: Generated new TSK 0
I20260808 18:27:24.227882 8734 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:27:24.263336 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41041
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:24.393676 8747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:24.393909 8747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:24.393951 8747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:24.397979 8747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:24.398033 8747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:24.398124 8747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:27:24.403209 8747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41041
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:24.404182 8747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:24.405164 8747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:24.414678 8760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:24.414741 8747 server_base.cc:1034] running on GCE node
W20260808 18:27:24.414857 8759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:24.414685 8764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:24.415357 8747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:24.415941 8747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:24.417126 8747 hybrid_clock.cc:648] HybridClock initialized: now 1786213644417110 us; error 36 us; skew 500 ppm
Aug 08 18:27:24 dist-test-slave-jztv krb5kdc[8592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213644, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:24.419822 8747 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:27:24.420619 8747 webserver.cc:460] Webserver started at http://127.7.164.65:33463/ using document root <none> and password file <none>
I20260808 18:27:24.421165 8747 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:24.421257 8747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:24.421487 8747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:24.423496 8747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "3c5fe9431b0d4b5d8dad522f3338e368"
format_stamp: "Formatted at 2026-08-08 18:27:24 on dist-test-slave-jztv"
server_key: "71b65c2ada81d243be379e4a48e3da33"
server_key_iv: "4d808bc3b68da77442c8f70dbe96ec6b"
server_key_version: "encryptionkey@0"
I20260808 18:27:24.424072 8747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "3c5fe9431b0d4b5d8dad522f3338e368"
format_stamp: "Formatted at 2026-08-08 18:27:24 on dist-test-slave-jztv"
server_key: "71b65c2ada81d243be379e4a48e3da33"
server_key_iv: "4d808bc3b68da77442c8f70dbe96ec6b"
server_key_version: "encryptionkey@0"
I20260808 18:27:24.427608 8747 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:27:24.429885 8773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:24.431100 8747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:24.431236 8747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "3c5fe9431b0d4b5d8dad522f3338e368"
format_stamp: "Formatted at 2026-08-08 18:27:24 on dist-test-slave-jztv"
server_key: "71b65c2ada81d243be379e4a48e3da33"
server_key_iv: "4d808bc3b68da77442c8f70dbe96ec6b"
server_key_version: "encryptionkey@0"
I20260808 18:27:24.431348 8747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:24.462285 8747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:24.465262 8747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:24.465479 8747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:24.466100 8747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:24.466962 8747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:24.467046 8747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:24.467118 8747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:24.467161 8747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:24.476292 8747 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:41217
I20260808 18:27:24.476321 8935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:41217 every 8 connection(s)
I20260808 18:27:24.477221 8747 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:27:24.480967 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 8747
I20260808 18:27:24.481055 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:27:24.481302 7825 external_mini_cluster.cc:1428] Setting key 5b9c7600f0abf869941db46062c9f019
Aug 08 18:27:24 dist-test-slave-jztv krb5kdc[8592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213644, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:24.492153 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:24.479040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:48405 (local address 127.7.164.126:41041)
0808 18:27:24.479181 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:24.479185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:24.480063 (+ 878us) server_negotiation.cc:407] Connection header received
0808 18:27:24.481519 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:24.481523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:24.481583 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:24.481697 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:24.483177 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.483708 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.485306 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.485731 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.485855 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:24.488226 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:24.488233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:24.488239 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:24.488275 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:24.489956 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:24.490532 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:24.490538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:24.490543 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:24.490642 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:24.491000 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:24.491004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:24.491006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:24.491244 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:24.491357 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:24.491854 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 18:27:24.491986 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:27:24.492880 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:24.479439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41041 (local address 127.7.164.65:48405)
0808 18:27:24.479900 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:24.479943 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:24.481270 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:24.481866 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:24.481873 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:24.482387 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:24.483015 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:24.483037 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.483843 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.483851 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:24.485174 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:24.485184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.485883 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.485886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:24.486022 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:24.486647 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:24.486663 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:24.488057 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:24.490093 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:24.490097 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:24.490108 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:24.490420 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:24.490751 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:24.490753 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:24.490754 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:24.490891 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:24.491364 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:24.491369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:24.491595 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 18:27:24.491737 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 18:27:24.491969 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":122,"threads_started":1}
I20260808 18:27:24.494163 8937 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41041
I20260808 18:27:24.494417 8937 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:24.494938 8937 heartbeater.cc:511] Master 127.7.164.126:41041 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
I20260808 18:27:24.496598 8653 ts_manager.cc:194] Registered new tserver with Master: 3c5fe9431b0d4b5d8dad522f3338e368 (127.7.164.65:41217)
I20260808 18:27:24.498279 8653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:48405
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:27:24.512603 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41041
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:24.623729 8944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:24.623975 8944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:24.624017 8944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:24.627907 8944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:24.627957 8944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:24.628014 8944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:27:24.633121 8944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41041
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:24.634109 8944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:24.635224 8944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:24.644991 8951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:24.645030 8952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:24.645031 8955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:24.645295 8944 server_base.cc:1034] running on GCE node
I20260808 18:27:24.645737 8944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:24.646405 8944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:24.647594 8944 hybrid_clock.cc:648] HybridClock initialized: now 1786213644647579 us; error 24 us; skew 500 ppm
Aug 08 18:27:24 dist-test-slave-jztv krb5kdc[8592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213644, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:24.650300 8944 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:27:24.651196 8944 webserver.cc:460] Webserver started at http://127.7.164.66:41185/ using document root <none> and password file <none>
I20260808 18:27:24.651753 8944 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:24.651815 8944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:24.652061 8944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:24.654062 8944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "85cc2086930b4fd4a881cf03526e39b7"
format_stamp: "Formatted at 2026-08-08 18:27:24 on dist-test-slave-jztv"
server_key: "cc986a3a147810d15d2e54242855aea3"
server_key_iv: "67fb50f9fc13898d93d0609d7b4e280b"
server_key_version: "encryptionkey@0"
I20260808 18:27:24.654692 8944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "85cc2086930b4fd4a881cf03526e39b7"
format_stamp: "Formatted at 2026-08-08 18:27:24 on dist-test-slave-jztv"
server_key: "cc986a3a147810d15d2e54242855aea3"
server_key_iv: "67fb50f9fc13898d93d0609d7b4e280b"
server_key_version: "encryptionkey@0"
I20260808 18:27:24.658422 8944 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:27:24.660827 8965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:24.662066 8944 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:24.662206 8944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "85cc2086930b4fd4a881cf03526e39b7"
format_stamp: "Formatted at 2026-08-08 18:27:24 on dist-test-slave-jztv"
server_key: "cc986a3a147810d15d2e54242855aea3"
server_key_iv: "67fb50f9fc13898d93d0609d7b4e280b"
server_key_version: "encryptionkey@0"
I20260808 18:27:24.662375 8944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:24.672047 8944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:24.674996 8944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:24.675206 8944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:24.675796 8944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:24.676651 8944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:24.676724 8944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:24.676796 8944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:24.676848 8944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:24.686271 8944 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:43875
I20260808 18:27:24.686301 9136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:43875 every 8 connection(s)
I20260808 18:27:24.687213 8944 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:27:24.689056 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 8944
I20260808 18:27:24.689183 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:27:24.690659 7825 external_mini_cluster.cc:1428] Setting key e6b240103e523afb77047e0e027f8489
Aug 08 18:27:24 dist-test-slave-jztv krb5kdc[8592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213644, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
I20260808 18:27:24.706146 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:24.690069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:59361 (local address 127.7.164.126:41041)
0808 18:27:24.691555 (+ 1486us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:24.691560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:24.691580 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:27:24.692949 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:24.692954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:24.693014 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:24.693101 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:24.696099 (+ 2998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.697011 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.698914 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.699140 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.699230 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:24.702548 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:24.702552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:24.702555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:24.702588 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:24.704019 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:24.704559 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:24.704562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:24.704564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:24.704650 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:24.705012 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:24.705015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:24.705017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:24.705211 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:24.705321 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:24.705898 (+ 577us) server_negotiation.cc:299] Negotiation successful
0808 18:27:24.706020 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1355}
I20260808 18:27:24.707260 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:24.690358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41041 (local address 127.7.164.66:59361)
0808 18:27:24.691036 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:24.691096 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:24.692732 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:24.693301 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:24.693310 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:24.693859 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:24.694631 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:24.694655 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.697182 (+ 2527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.697186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:24.698773 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:24.698789 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.699312 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.699320 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:24.699575 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:24.700294 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:24.700323 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:24.702360 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:24.704146 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:24.704152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:24.704168 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:24.704436 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:24.704749 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:24.704752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:24.704754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:24.704903 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:24.705322 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:24.705330 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:24.705588 (+ 258us) client_negotiation.cc:769] Sending connection context
0808 18:27:24.705822 (+ 234us) client_negotiation.cc:240] Negotiation successful
0808 18:27:24.706132 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":158,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
I20260808 18:27:24.708760 9137 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41041
I20260808 18:27:24.709031 9137 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:24.709550 9137 heartbeater.cc:511] Master 127.7.164.126:41041 requested a full tablet report, sending...
I20260808 18:27:24.710727 8653 ts_manager.cc:194] Registered new tserver with Master: 85cc2086930b4fd4a881cf03526e39b7 (127.7.164.66:43875)
I20260808 18:27:24.711529 8653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:59361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:27:24.725237 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41041
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:24.838004 9144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:24.838322 9144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:24.838370 9144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:24.842386 9144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:24.842435 9144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:24.842495 9144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:27:24.847532 9144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41041
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:24.848513 9144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:24.849586 9144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:24.858985 9157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:24.858986 9155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:24.859022 9159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:24.859380 9144 server_base.cc:1034] running on GCE node
I20260808 18:27:24.859795 9144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:24.860349 9144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:24.861512 9144 hybrid_clock.cc:648] HybridClock initialized: now 1786213644861490 us; error 33 us; skew 500 ppm
Aug 08 18:27:24 dist-test-slave-jztv krb5kdc[8592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213644, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:24.864284 9144 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:27:24.865180 9144 webserver.cc:460] Webserver started at http://127.7.164.67:45823/ using document root <none> and password file <none>
I20260808 18:27:24.865784 9144 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:24.865875 9144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:24.866113 9144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:24.868223 9144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "55bf23e09fd14a2c88f9e5a821fdcb93"
format_stamp: "Formatted at 2026-08-08 18:27:24 on dist-test-slave-jztv"
server_key: "f024a90aeaff5c67ffc5994f4f7495c6"
server_key_iv: "082682a991c6101a476d34d565ec713d"
server_key_version: "encryptionkey@0"
I20260808 18:27:24.868924 9144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "55bf23e09fd14a2c88f9e5a821fdcb93"
format_stamp: "Formatted at 2026-08-08 18:27:24 on dist-test-slave-jztv"
server_key: "f024a90aeaff5c67ffc5994f4f7495c6"
server_key_iv: "082682a991c6101a476d34d565ec713d"
server_key_version: "encryptionkey@0"
I20260808 18:27:24.872802 9144 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:27:24.875221 9169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:24.876327 9144 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:24.876468 9144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "55bf23e09fd14a2c88f9e5a821fdcb93"
format_stamp: "Formatted at 2026-08-08 18:27:24 on dist-test-slave-jztv"
server_key: "f024a90aeaff5c67ffc5994f4f7495c6"
server_key_iv: "082682a991c6101a476d34d565ec713d"
server_key_version: "encryptionkey@0"
I20260808 18:27:24.876582 9144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:24.893324 9144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:24.896783 9144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:24.896979 9144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:24.897555 9144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:24.898452 9144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:24.898520 9144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:24.898586 9144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:24.898633 9144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:24.907754 9144 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:39063
I20260808 18:27:24.907796 9342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:39063 every 8 connection(s)
I20260808 18:27:24.908618 9144 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:27:24.912075 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 9144
I20260808 18:27:24.912173 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:27:24.912461 7825 external_mini_cluster.cc:1428] Setting key da0e8320c0d5764dd5efb365655ebfec
Aug 08 18:27:24 dist-test-slave-jztv krb5kdc[8592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213644, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:24.921871 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:24.910341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:51201 (local address 127.7.164.126:41041)
0808 18:27:24.910492 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:24.910496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:24.911252 (+ 756us) server_negotiation.cc:407] Connection header received
0808 18:27:24.912753 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:24.912757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:24.912817 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:24.912922 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:24.914327 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.914940 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.916161 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.916365 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.916439 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:24.918712 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:24.918715 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:24.918717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:24.918741 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:24.920192 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:24.920624 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:24.920626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:24.920627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:24.920701 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:24.920997 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:24.920999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:24.921000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:24.921163 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:24.921235 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:24.921661 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 18:27:24.921754 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:27:24.922739 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:24.910670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41041 (local address 127.7.164.67:51201)
0808 18:27:24.911102 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:24.911145 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:24.912526 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:24.913127 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:24.913136 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:24.913515 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:24.914088 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:24.914108 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.915064 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.915067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:24.916055 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:24.916066 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.916498 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.916500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:24.916632 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:24.917145 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:24.917160 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:24.918561 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:24.920292 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:24.920296 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:24.920307 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:24.920538 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:24.920780 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:24.920782 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:24.920783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:24.920908 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:24.921250 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:24.921254 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:24.921462 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 18:27:24.921614 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 18:27:24.921846 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":110,"threads_started":1}
I20260808 18:27:24.924032 9345 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41041
I20260808 18:27:24.924307 9345 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:24.924853 9345 heartbeater.cc:511] Master 127.7.164.126:41041 requested a full tablet report, sending...
I20260808 18:27:24.926038 8653 ts_manager.cc:194] Registered new tserver with Master: 55bf23e09fd14a2c88f9e5a821fdcb93 (127.7.164.67:39063)
I20260808 18:27:24.926544 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:27:24.926775 8653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:51201
I20260808 18:27:24.937294 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:24.929942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50592 (local address 127.7.164.126:41041)
0808 18:27:24.930098 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:24.930103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:24.930267 (+ 164us) server_negotiation.cc:407] Connection header received
0808 18:27:24.930436 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:24.930439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:24.930490 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:24.930575 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:24.931246 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.931736 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.932561 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.932776 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.932882 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:24.934193 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:24.934197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:24.934200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:24.934252 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:24.935654 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:24.936071 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:24.936076 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:24.936077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:24.936156 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:24.936432 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:24.936437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:24.936439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:24.936644 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:24.936785 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:24.937027 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 18:27:24.937142 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 18:27:24.941920 8653 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:27:24.944947 8653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:27:24.959914 9372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:24.954976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47098 (local address 127.7.164.66:43875)
0808 18:27:24.955537 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:24.955549 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:24.955570 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:27:24.955641 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:24.955647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:24.955821 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:24.955980 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:24.956161 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.957154 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.958453 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.959159 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.959287 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:24.959458 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:24.959636 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 18:27:24.959744 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":135,"threads_started":1}
I20260808 18:27:24.959924 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:24.953428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:39063 (local address 127.0.0.1:46988)
0808 18:27:24.953815 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:24.953866 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:24.954010 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:24.954766 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:24.954772 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:24.954797 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:24.954959 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:24.954966 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.956486 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.956489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:24.958266 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:24.958277 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.959408 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.959411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:24.959508 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:24.959531 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:27:24.959586 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 18:27:24.959743 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":268,"spinlock_wait_cycles":3200,"thread_start_us":102,"threads_started":1}
I20260808 18:27:24.960256 9368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:24.954361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55308 (local address 127.7.164.65:41217)
0808 18:27:24.954761 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:24.954779 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:24.954804 (+ 25us) server_negotiation.cc:407] Connection header received
0808 18:27:24.954880 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:24.954889 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:24.955072 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:24.955226 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:24.955490 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.956600 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.958413 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.959244 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.959381 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:24.959524 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:24.960009 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 18:27:24.960110 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":109,"threads_started":1}
I20260808 18:27:24.960297 9364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:24.953586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46988 (local address 127.7.164.67:39063)
0808 18:27:24.953971 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:24.953992 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:24.954017 (+ 25us) server_negotiation.cc:407] Connection header received
0808 18:27:24.954357 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:24.954363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:24.954577 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:24.955145 (+ 568us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:24.955309 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.956356 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.958412 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.959244 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.959888 (+ 644us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:24.960031 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:24.960075 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 18:27:24.960153 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":162,"threads_started":1}
I20260808 18:27:24.960394 9366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:24.954181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:41217 (local address 127.0.0.1:55308)
0808 18:27:24.954431 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:24.954445 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:24.954550 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:24.955204 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:24.955207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:24.955224 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:24.955335 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:24.955346 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.956736 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.956739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:24.958275 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:24.958282 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.959586 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.959590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:24.959666 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:24.959693 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:27:24.959757 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 18:27:24.959813 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":323,"thread_start_us":119,"threads_started":1}
I20260808 18:27:24.959909 9369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:24.954550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:43875 (local address 127.0.0.1:47098)
0808 18:27:24.954755 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:24.954768 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:24.954859 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:24.955950 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:24.955953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:24.955973 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:24.956061 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:24.956066 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.957320 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.957323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:24.958309 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:24.958320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:24.959289 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:24.959293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:24.959415 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:24.959454 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 18:27:24.959601 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 18:27:24.959760 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":144,"spinlock_wait_cycles":896,"thread_start_us":94,"threads_started":1}
I20260808 18:27:24.963388 9035 tablet_service.cc:1467] Processing CreateTablet for tablet 2381e76fe7f244e0b2c732062c07311e (DEFAULT_TABLE table=test-table [id=29f251da75684580b139a01fa00e14eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:24.963395 8840 tablet_service.cc:1467] Processing CreateTablet for tablet 2381e76fe7f244e0b2c732062c07311e (DEFAULT_TABLE table=test-table [id=29f251da75684580b139a01fa00e14eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:24.963397 9238 tablet_service.cc:1467] Processing CreateTablet for tablet 2381e76fe7f244e0b2c732062c07311e (DEFAULT_TABLE table=test-table [id=29f251da75684580b139a01fa00e14eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:24.964424 9035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2381e76fe7f244e0b2c732062c07311e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:24.964440 8840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2381e76fe7f244e0b2c732062c07311e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:24.964440 9238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2381e76fe7f244e0b2c732062c07311e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:24.970789 9373 tablet_bootstrap.cc:492] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7: Bootstrap starting.
I20260808 18:27:24.972097 9375 tablet_bootstrap.cc:492] T 2381e76fe7f244e0b2c732062c07311e P 3c5fe9431b0d4b5d8dad522f3338e368: Bootstrap starting.
I20260808 18:27:24.973757 9375 tablet_bootstrap.cc:654] T 2381e76fe7f244e0b2c732062c07311e P 3c5fe9431b0d4b5d8dad522f3338e368: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:24.974056 9373 tablet_bootstrap.cc:654] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:24.974565 9374 tablet_bootstrap.cc:492] T 2381e76fe7f244e0b2c732062c07311e P 55bf23e09fd14a2c88f9e5a821fdcb93: Bootstrap starting.
I20260808 18:27:24.974998 9375 log.cc:824] T 2381e76fe7f244e0b2c732062c07311e P 3c5fe9431b0d4b5d8dad522f3338e368: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:24.975001 9373 log.cc:824] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:24.977027 9373 tablet_bootstrap.cc:492] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7: No bootstrap required, opened a new log
I20260808 18:27:24.977033 9374 tablet_bootstrap.cc:654] T 2381e76fe7f244e0b2c732062c07311e P 55bf23e09fd14a2c88f9e5a821fdcb93: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:24.977108 9375 tablet_bootstrap.cc:492] T 2381e76fe7f244e0b2c732062c07311e P 3c5fe9431b0d4b5d8dad522f3338e368: No bootstrap required, opened a new log
I20260808 18:27:24.977311 9375 ts_tablet_manager.cc:1397] T 2381e76fe7f244e0b2c732062c07311e P 3c5fe9431b0d4b5d8dad522f3338e368: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260808 18:27:24.977334 9373 ts_tablet_manager.cc:1397] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 18:27:24.977815 9374 log.cc:824] T 2381e76fe7f244e0b2c732062c07311e P 55bf23e09fd14a2c88f9e5a821fdcb93: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:24.979876 9374 tablet_bootstrap.cc:492] T 2381e76fe7f244e0b2c732062c07311e P 55bf23e09fd14a2c88f9e5a821fdcb93: No bootstrap required, opened a new log
I20260808 18:27:24.980067 9374 ts_tablet_manager.cc:1397] T 2381e76fe7f244e0b2c732062c07311e P 55bf23e09fd14a2c88f9e5a821fdcb93: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:27:24.980799 9373 raft_consensus.cc:348] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55bf23e09fd14a2c88f9e5a821fdcb93" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39063 } } peers { permanent_uuid: "3c5fe9431b0d4b5d8dad522f3338e368" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41217 } } peers { permanent_uuid: "85cc2086930b4fd4a881cf03526e39b7" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43875 } }
I20260808 18:27:24.981051 9373 raft_consensus.cc:374] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:24.981105 9373 raft_consensus.cc:729] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85cc2086930b4fd4a881cf03526e39b7, State: Initialized, Role: FOLLOWER
I20260808 18:27:24.981235 9375 raft_consensus.cc:348] T 2381e76fe7f244e0b2c732062c07311e P 3c5fe9431b0d4b5d8dad522f3338e368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55bf23e09fd14a2c88f9e5a821fdcb93" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39063 } } peers { permanent_uuid: "3c5fe9431b0d4b5d8dad522f3338e368" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41217 } } peers { permanent_uuid: "85cc2086930b4fd4a881cf03526e39b7" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43875 } }
I20260808 18:27:24.981484 9375 raft_consensus.cc:374] T 2381e76fe7f244e0b2c732062c07311e P 3c5fe9431b0d4b5d8dad522f3338e368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:24.981539 9375 raft_consensus.cc:729] T 2381e76fe7f244e0b2c732062c07311e P 3c5fe9431b0d4b5d8dad522f3338e368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c5fe9431b0d4b5d8dad522f3338e368, State: Initialized, Role: FOLLOWER
I20260808 18:27:24.981600 9373 consensus_queue.cc:260] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55bf23e09fd14a2c88f9e5a821fdcb93" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39063 } } peers { permanent_uuid: "3c5fe9431b0d4b5d8dad522f3338e368" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41217 } } peers { permanent_uuid: "85cc2086930b4fd4a881cf03526e39b7" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43875 } }
I20260808 18:27:24.981997 9375 consensus_queue.cc:260] T 2381e76fe7f244e0b2c732062c07311e P 3c5fe9431b0d4b5d8dad522f3338e368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55bf23e09fd14a2c88f9e5a821fdcb93" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39063 } } peers { permanent_uuid: "3c5fe9431b0d4b5d8dad522f3338e368" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41217 } } peers { permanent_uuid: "85cc2086930b4fd4a881cf03526e39b7" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43875 } }
I20260808 18:27:24.982366 9137 heartbeater.cc:503] Master 127.7.164.126:41041 was elected leader, sending a full tablet report...
I20260808 18:27:24.982668 9373 ts_tablet_manager.cc:1428] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:27:24.982698 8937 heartbeater.cc:503] Master 127.7.164.126:41041 was elected leader, sending a full tablet report...
I20260808 18:27:24.982960 9375 ts_tablet_manager.cc:1428] T 2381e76fe7f244e0b2c732062c07311e P 3c5fe9431b0d4b5d8dad522f3338e368: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260808 18:27:24.984714 9374 raft_consensus.cc:348] T 2381e76fe7f244e0b2c732062c07311e P 55bf23e09fd14a2c88f9e5a821fdcb93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55bf23e09fd14a2c88f9e5a821fdcb93" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39063 } } peers { permanent_uuid: "3c5fe9431b0d4b5d8dad522f3338e368" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41217 } } peers { permanent_uuid: "85cc2086930b4fd4a881cf03526e39b7" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43875 } }
I20260808 18:27:24.984949 9374 raft_consensus.cc:374] T 2381e76fe7f244e0b2c732062c07311e P 55bf23e09fd14a2c88f9e5a821fdcb93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:24.985008 9374 raft_consensus.cc:729] T 2381e76fe7f244e0b2c732062c07311e P 55bf23e09fd14a2c88f9e5a821fdcb93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55bf23e09fd14a2c88f9e5a821fdcb93, State: Initialized, Role: FOLLOWER
I20260808 18:27:24.985451 9374 consensus_queue.cc:260] T 2381e76fe7f244e0b2c732062c07311e P 55bf23e09fd14a2c88f9e5a821fdcb93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55bf23e09fd14a2c88f9e5a821fdcb93" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39063 } } peers { permanent_uuid: "3c5fe9431b0d4b5d8dad522f3338e368" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41217 } } peers { permanent_uuid: "85cc2086930b4fd4a881cf03526e39b7" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43875 } }
I20260808 18:27:24.986063 9345 heartbeater.cc:503] Master 127.7.164.126:41041 was elected leader, sending a full tablet report...
I20260808 18:27:24.986414 9374 ts_tablet_manager.cc:1428] T 2381e76fe7f244e0b2c732062c07311e P 55bf23e09fd14a2c88f9e5a821fdcb93: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260808 18:27:25.159548 9346 tablet.cc:2369] T 2381e76fe7f244e0b2c732062c07311e P 55bf23e09fd14a2c88f9e5a821fdcb93: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:27:25.188375 9139 tablet.cc:2369] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:27:25.228569 8938 tablet.cc:2369] T 2381e76fe7f244e0b2c732062c07311e P 3c5fe9431b0d4b5d8dad522f3338e368: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:27:25.259038 9385 raft_consensus.cc:482] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:27:25.259240 9385 raft_consensus.cc:504] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55bf23e09fd14a2c88f9e5a821fdcb93" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39063 } } peers { permanent_uuid: "3c5fe9431b0d4b5d8dad522f3338e368" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41217 } } peers { permanent_uuid: "85cc2086930b4fd4a881cf03526e39b7" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43875 } }
I20260808 18:27:25.260332 9385 leader_election.cc:288] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55bf23e09fd14a2c88f9e5a821fdcb93 (127.7.164.67:39063), 3c5fe9431b0d4b5d8dad522f3338e368 (127.7.164.65:41217)
I20260808 18:27:25.264328 9364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:25.260728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:33119 (local address 127.7.164.67:39063)
0808 18:27:25.260862 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:25.260868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:25.260973 (+ 105us) server_negotiation.cc:407] Connection header received
0808 18:27:25.261197 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:25.261204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:25.261268 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:25.261368 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:25.261713 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:25.262279 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:25.263366 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:25.263765 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:25.263904 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:25.263962 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:25.264136 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 18:27:25.264203 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:27:25.264351 9391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:25.260609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:39063 (local address 127.7.164.66:33119)
0808 18:27:25.260863 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:25.260881 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:25.261000 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:25.261409 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:25.261413 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:25.261463 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:25.261584 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:25.261589 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:25.262400 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:25.262404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:25.263242 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:25.263252 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:25.263889 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:25.263893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:25.263998 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:25.264022 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:27:25.264121 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 18:27:25.264194 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":20,"spinlock_wait_cycles":32256,"thread_start_us":92,"threads_started":1}
I20260808 18:27:25.264530 9368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:25.261142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:33667 (local address 127.7.164.65:41217)
0808 18:27:25.261291 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:25.261297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:25.261418 (+ 121us) server_negotiation.cc:407] Connection header received
0808 18:27:25.261605 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:25.261608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:25.261660 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:25.261745 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:25.262043 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:25.262548 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:25.263581 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:25.263965 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:25.264102 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:25.264163 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:25.264344 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 18:27:25.264416 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 18:27:25.264554 9392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:25.261008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:41217 (local address 127.7.164.66:33667)
0808 18:27:25.261302 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:25.261312 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:25.261474 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:25.261757 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:25.261760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:25.261776 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:25.261908 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:25.261914 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:25.262666 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:25.262670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:25.263445 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:25.263453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:25.264096 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:25.264100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:25.264210 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:25.264237 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:27:25.264326 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 18:27:25.264397 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":1,"spinlock_wait_cycles":6656,"thread_start_us":90,"threads_started":1}
I20260808 18:27:25.265254 8867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2381e76fe7f244e0b2c732062c07311e" candidate_uuid: "85cc2086930b4fd4a881cf03526e39b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c5fe9431b0d4b5d8dad522f3338e368" is_pre_election: true
I20260808 18:27:25.265257 9275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2381e76fe7f244e0b2c732062c07311e" candidate_uuid: "85cc2086930b4fd4a881cf03526e39b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55bf23e09fd14a2c88f9e5a821fdcb93" is_pre_election: true
I20260808 18:27:25.265609 9275 raft_consensus.cc:2454] T 2381e76fe7f244e0b2c732062c07311e P 55bf23e09fd14a2c88f9e5a821fdcb93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85cc2086930b4fd4a881cf03526e39b7 in term 0.
I20260808 18:27:25.265609 8867 raft_consensus.cc:2454] T 2381e76fe7f244e0b2c732062c07311e P 3c5fe9431b0d4b5d8dad522f3338e368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85cc2086930b4fd4a881cf03526e39b7 in term 0.
I20260808 18:27:25.266111 8972 leader_election.cc:302] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c5fe9431b0d4b5d8dad522f3338e368, 85cc2086930b4fd4a881cf03526e39b7; no voters:
I20260808 18:27:25.266410 9385 raft_consensus.cc:2781] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:27:25.266528 9385 raft_consensus.cc:482] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:27:25.266588 9385 raft_consensus.cc:3037] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:25.267658 9385 raft_consensus.cc:504] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55bf23e09fd14a2c88f9e5a821fdcb93" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39063 } } peers { permanent_uuid: "3c5fe9431b0d4b5d8dad522f3338e368" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41217 } } peers { permanent_uuid: "85cc2086930b4fd4a881cf03526e39b7" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43875 } }
I20260808 18:27:25.268002 9385 leader_election.cc:288] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [CANDIDATE]: Term 1 election: Requested vote from peers 55bf23e09fd14a2c88f9e5a821fdcb93 (127.7.164.67:39063), 3c5fe9431b0d4b5d8dad522f3338e368 (127.7.164.65:41217)
I20260808 18:27:25.268381 9275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2381e76fe7f244e0b2c732062c07311e" candidate_uuid: "85cc2086930b4fd4a881cf03526e39b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55bf23e09fd14a2c88f9e5a821fdcb93"
I20260808 18:27:25.268446 8867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2381e76fe7f244e0b2c732062c07311e" candidate_uuid: "85cc2086930b4fd4a881cf03526e39b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c5fe9431b0d4b5d8dad522f3338e368"
I20260808 18:27:25.268519 9275 raft_consensus.cc:3037] T 2381e76fe7f244e0b2c732062c07311e P 55bf23e09fd14a2c88f9e5a821fdcb93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:25.268574 8867 raft_consensus.cc:3037] T 2381e76fe7f244e0b2c732062c07311e P 3c5fe9431b0d4b5d8dad522f3338e368 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:25.269553 9275 raft_consensus.cc:2454] T 2381e76fe7f244e0b2c732062c07311e P 55bf23e09fd14a2c88f9e5a821fdcb93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85cc2086930b4fd4a881cf03526e39b7 in term 1.
I20260808 18:27:25.269625 8867 raft_consensus.cc:2454] T 2381e76fe7f244e0b2c732062c07311e P 3c5fe9431b0d4b5d8dad522f3338e368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85cc2086930b4fd4a881cf03526e39b7 in term 1.
I20260808 18:27:25.269858 8972 leader_election.cc:302] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55bf23e09fd14a2c88f9e5a821fdcb93, 85cc2086930b4fd4a881cf03526e39b7; no voters:
I20260808 18:27:25.270030 9385 raft_consensus.cc:2781] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:25.270282 9385 raft_consensus.cc:686] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [term 1 LEADER]: Becoming Leader. State: Replica: 85cc2086930b4fd4a881cf03526e39b7, State: Running, Role: LEADER
I20260808 18:27:25.270639 9385 consensus_queue.cc:237] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55bf23e09fd14a2c88f9e5a821fdcb93" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39063 } } peers { permanent_uuid: "3c5fe9431b0d4b5d8dad522f3338e368" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41217 } } peers { permanent_uuid: "85cc2086930b4fd4a881cf03526e39b7" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43875 } }
I20260808 18:27:25.273944 8653 catalog_manager.cc:5495] T 2381e76fe7f244e0b2c732062c07311e P 85cc2086930b4fd4a881cf03526e39b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85cc2086930b4fd4a881cf03526e39b7 (127.7.164.66). New cstate: current_term: 1 leader_uuid: "85cc2086930b4fd4a881cf03526e39b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55bf23e09fd14a2c88f9e5a821fdcb93" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c5fe9431b0d4b5d8dad522f3338e368" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85cc2086930b4fd4a881cf03526e39b7" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43875 } health_report { overall_health: HEALTHY } } }
Aug 08 18:27:25 dist-test-slave-jztv krb5kdc[8592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.65@KRBTEST.COM
I20260808 18:27:25.330063 9368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:25.316740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39710 (local address 127.7.164.65:41217)
0808 18:27:25.316890 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:25.316895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:25.317139 (+ 244us) server_negotiation.cc:407] Connection header received
0808 18:27:25.317300 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:25.317303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:25.317358 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:25.317446 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:25.318169 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:25.318744 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:25.319623 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:25.319851 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:25.319961 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:25.326520 (+ 6559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:25.326535 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:25.326552 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:25.326579 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:25.328517 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:25.328910 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:25.328915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:25.328917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:25.329003 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:25.329229 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:25.329231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:25.329232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:25.329509 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:25.329675 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:25.329832 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 18:27:25.329936 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 18:27:25 dist-test-slave-jztv krb5kdc[8592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213645, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:27:25 dist-test-slave-jztv krb5kdc[8592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213645, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.164.65@KRBTEST.COM
I20260808 18:27:25.358845 9368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:25.350003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39726 (local address 127.7.164.65:41217)
0808 18:27:25.350208 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:25.350256 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:25.350350 (+ 94us) server_negotiation.cc:407] Connection header received
0808 18:27:25.350526 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:25.350529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:25.350576 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:25.350651 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:25.351387 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:25.351916 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:25.352786 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:25.353000 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:25.353073 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:25.355569 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:25.355573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:25.355575 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:25.355599 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:25.357032 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:25.357482 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:25.357485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:25.357487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:25.357562 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:25.357854 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:25.357859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:25.357860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:25.358037 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:25.358179 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:25.358543 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 18:27:25.358673 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
Aug 08 18:27:25 dist-test-slave-jztv krb5kdc[8592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:27:25 dist-test-slave-jztv krb5kdc[8592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.65@KRBTEST.COM
I20260808 18:27:25.385737 9368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:25.377323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39728 (local address 127.7.164.65:41217)
0808 18:27:25.377524 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:25.377528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:25.377595 (+ 67us) server_negotiation.cc:407] Connection header received
0808 18:27:25.377708 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:25.377711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:25.377756 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:25.377826 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:25.378648 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:25.379155 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:25.380093 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:25.380298 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:25.380375 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:25.382732 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:25.382735 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:25.382737 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:25.382761 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:25.384201 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:25.384586 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:25.384588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:25.384590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:25.384669 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:25.384966 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:25.384969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:25.384970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:25.385160 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:25.385280 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:25.385503 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 18:27:25.385612 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260808 18:27:25.387233 8814 tablet_service.cc:2908] Rejecting scan request for tablet 2381e76fe7f244e0b2c732062c07311e: Uninitialized: clean time has not yet been initialized
I20260808 18:27:25.389698 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 8747
I20260808 18:27:25.408087 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 8944
I20260808 18:27:25.421334 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 9144
I20260808 18:27:25.440560 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 8607
2026-08-08T18:27:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2794 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:25 dist-test-slave-jztv krb5kdc[9415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:25 dist-test-slave-jztv krb5kdc[9415](info): set up 2 sockets
Aug 08 18:27:25 dist-test-slave-jztv krb5kdc[9415](info): commencing operation
krb5kdc: starting...
W20260808 18:27:26.679306 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:27:26 dist-test-slave-jztv krb5kdc[9415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:27:26.810776 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:37137
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:41817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:37137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:26.925864 9432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:26.926121 9432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:26.926159 9432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:26.930128 9432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:26.930184 9432 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 18:27:26.930202 9432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:26.930234 9432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:26.930250 9432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:26.930265 9432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:26.935384 9432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:41817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:37137
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:37137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:26.936343 9432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:26.937361 9432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:27:26.945611 9432 server_base.cc:1034] running on GCE node
W20260808 18:27:26.945559 9440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:26.945576 9437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:26.945822 9442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:26.946403 9432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:26.947474 9432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:26.948639 9432 hybrid_clock.cc:648] HybridClock initialized: now 1786213646948621 us; error 29 us; skew 500 ppm
Aug 08 18:27:26 dist-test-slave-jztv krb5kdc[9415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213646, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:26.951408 9432 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:27:26.952298 9432 webserver.cc:460] Webserver started at http://127.7.164.126:36969/ using document root <none> and password file <none>
I20260808 18:27:26.952838 9432 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:26.952898 9432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:26.953132 9432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:26.955152 9432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "cda07c0b897d45f18b513897b9e8ddd9"
format_stamp: "Formatted at 2026-08-08 18:27:26 on dist-test-slave-jztv"
server_key: "2ea66cfaa3d98626b18faf8ceff35b21"
server_key_iv: "4b04644bc479d58fbc5f2c049f0c1900"
server_key_version: "encryptionkey@0"
I20260808 18:27:26.955710 9432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "cda07c0b897d45f18b513897b9e8ddd9"
format_stamp: "Formatted at 2026-08-08 18:27:26 on dist-test-slave-jztv"
server_key: "2ea66cfaa3d98626b18faf8ceff35b21"
server_key_iv: "4b04644bc479d58fbc5f2c049f0c1900"
server_key_version: "encryptionkey@0"
I20260808 18:27:26.959345 9432 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:27:26.961788 9450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:26.963196 9432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:27:26.963364 9432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "cda07c0b897d45f18b513897b9e8ddd9"
format_stamp: "Formatted at 2026-08-08 18:27:26 on dist-test-slave-jztv"
server_key: "2ea66cfaa3d98626b18faf8ceff35b21"
server_key_iv: "4b04644bc479d58fbc5f2c049f0c1900"
server_key_version: "encryptionkey@0"
I20260808 18:27:26.963502 9432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:26.979947 9432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:26.983191 9432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:26.983397 9432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:26.991422 9432 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:37137
I20260808 18:27:26.991430 9548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:37137 every 8 connection(s)
I20260808 18:27:26.992341 9432 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:26.995039 9549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:26.998199 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 9432
I20260808 18:27:26.998379 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:26.998637 7825 external_mini_cluster.cc:1428] Setting key 048c46d089f3ac0c9ba585a6c5d9710b
I20260808 18:27:27.000947 9549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9: Bootstrap starting.
I20260808 18:27:27.003523 9549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:27.004406 9549 log.cc:824] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:27.006656 9549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9: No bootstrap required, opened a new log
Aug 08 18:27:27 dist-test-slave-jztv krb5kdc[9415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:27.011684 9549 raft_consensus.cc:348] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda07c0b897d45f18b513897b9e8ddd9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37137 } }
I20260808 18:27:27.011984 9549 raft_consensus.cc:374] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:27.012061 9549 raft_consensus.cc:729] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cda07c0b897d45f18b513897b9e8ddd9, State: Initialized, Role: FOLLOWER
I20260808 18:27:27.012580 9549 consensus_queue.cc:260] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda07c0b897d45f18b513897b9e8ddd9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37137 } }
I20260808 18:27:27.012743 9549 raft_consensus.cc:388] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:27.012812 9549 raft_consensus.cc:482] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:27.012915 9549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:27.014103 9549 raft_consensus.cc:504] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda07c0b897d45f18b513897b9e8ddd9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37137 } }
I20260808 18:27:27.014498 9549 leader_election.cc:302] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cda07c0b897d45f18b513897b9e8ddd9; no voters:
I20260808 18:27:27.014739 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.000064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41658 (local address 127.7.164.126:37137)
0808 18:27:27.000475 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.000491 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.000527 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:27:27.001185 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.001203 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.001527 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.001797 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:27.002783 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.003841 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.004989 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.005558 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.005680 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.009043 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:27.009049 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:27.009065 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:27.009098 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:27.011360 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.011825 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.011831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.011836 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.011975 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.012343 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.012346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.012348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.012877 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:27.013074 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.013384 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.013693 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":126,"threads_started":1}
I20260808 18:27:27.014847 9549 leader_election.cc:288] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:27.015000 9558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:27.015225 9558 raft_consensus.cc:686] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [term 1 LEADER]: Becoming Leader. State: Replica: cda07c0b897d45f18b513897b9e8ddd9, State: Running, Role: LEADER
I20260808 18:27:27.015561 9558 consensus_queue.cc:237] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda07c0b897d45f18b513897b9e8ddd9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37137 } }
I20260808 18:27:27.016125 9549 sys_catalog.cc:564] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:27.018193 9562 sys_catalog.cc:455] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cda07c0b897d45f18b513897b9e8ddd9. Latest consensus state: current_term: 1 leader_uuid: "cda07c0b897d45f18b513897b9e8ddd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda07c0b897d45f18b513897b9e8ddd9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37137 } } }
I20260808 18:27:27.018368 9562 sys_catalog.cc:458] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:27.018720 9580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:27.018694 9561 sys_catalog.cc:455] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cda07c0b897d45f18b513897b9e8ddd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda07c0b897d45f18b513897b9e8ddd9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37137 } } }
I20260808 18:27:27.019202 9561 sys_catalog.cc:458] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:27.021653 9580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:27.027493 9580 catalog_manager.cc:1347] Generated new cluster ID: 07326ac2f93e49abb23f530fa05d1663
I20260808 18:27:27.027562 9580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:27.038837 9580 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:27.040089 9580 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:27.047809 9580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9: Generated new TSK 0
I20260808 18:27:27.048461 9580 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:27:27.086606 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:37137
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:41817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:27.198875 9587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:27.199126 9587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:27.199201 9587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:27.203267 9587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:27.203320 9587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:27.203384 9587 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:27:27.203447 9587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:27:27.208500 9587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:41817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.164.126:37137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:27.209474 9587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:27.210538 9587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:27.220466 9598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:27.220468 9600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:27.220516 9602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:27.220674 9587 server_base.cc:1034] running on GCE node
I20260808 18:27:27.221078 9587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:27.221623 9587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:27.222807 9587 hybrid_clock.cc:648] HybridClock initialized: now 1786213647222793 us; error 30 us; skew 500 ppm
Aug 08 18:27:27 dist-test-slave-jztv krb5kdc[9415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:27.225502 9587 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:27:27.226440 9587 webserver.cc:460] Webserver started at http://127.7.164.65:33183/ using document root <none> and password file <none>
I20260808 18:27:27.226940 9587 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:27.227005 9587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:27.227165 9587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:27.229069 9587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "647495edaa1f493b838a4e9687d6b948"
format_stamp: "Formatted at 2026-08-08 18:27:27 on dist-test-slave-jztv"
server_key: "b68e742730ea75de3d994b4e014c3ebc"
server_key_iv: "813386b8074278abaacf04e53d34da75"
server_key_version: "encryptionkey@0"
I20260808 18:27:27.229570 9587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "647495edaa1f493b838a4e9687d6b948"
format_stamp: "Formatted at 2026-08-08 18:27:27 on dist-test-slave-jztv"
server_key: "b68e742730ea75de3d994b4e014c3ebc"
server_key_iv: "813386b8074278abaacf04e53d34da75"
server_key_version: "encryptionkey@0"
I20260808 18:27:27.233157 9587 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:27:27.235571 9612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:27.236750 9587 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:27:27.236868 9587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "647495edaa1f493b838a4e9687d6b948"
format_stamp: "Formatted at 2026-08-08 18:27:27 on dist-test-slave-jztv"
server_key: "b68e742730ea75de3d994b4e014c3ebc"
server_key_iv: "813386b8074278abaacf04e53d34da75"
server_key_version: "encryptionkey@0"
I20260808 18:27:27.236970 9587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:27.248967 9587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:27.251753 9587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:27.251945 9587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:27.253400 9587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:27.253455 9587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:27.253530 9587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:27.253568 9587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:27:27 dist-test-slave-jztv krb5kdc[9415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:27.264717 9587 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:36909
I20260808 18:27:27.264896 9787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:36909 every 8 connection(s)
I20260808 18:27:27.265697 9587 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:27:27.268234 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.254169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:45493 (local address 127.7.164.126:37137)
0808 18:27:27.254348 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.254352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.255269 (+ 917us) server_negotiation.cc:407] Connection header received
0808 18:27:27.256546 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.256552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.256620 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.256687 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:27.258559 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.259527 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.260662 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.261032 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.261150 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.264160 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:27.264163 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:27.264166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:27.264194 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:27.265813 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.266473 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.266479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.266481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.266575 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.267021 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.267031 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.267033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.267286 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:27.267405 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.267951 (+ 546us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.268062 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260808 18:27:27.268955 9622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.254578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37137 (local address 127.7.164.65:45493)
0808 18:27:27.255084 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:27.255145 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:27.256275 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:27.257155 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:27.257167 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:27.257631 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:27.258392 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.258417 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.259653 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.259657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.260530 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.260540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.261341 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.261347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.261584 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.262255 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:27.262277 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:27.263989 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:27.265940 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:27.265946 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:27.265960 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:27.266362 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:27.266673 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:27.266677 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:27.266679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:27.266845 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:27.267408 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:27.267412 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:27.267716 (+ 304us) client_negotiation.cc:769] Sending connection context
0808 18:27:27.267864 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 18:27:27.268112 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":103,"threads_started":1}
I20260808 18:27:27.270208 9788 heartbeater.cc:347] Connected to a master server at 127.7.164.126:37137
I20260808 18:27:27.270392 9788 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:27.270891 9788 heartbeater.cc:511] Master 127.7.164.126:37137 requested a full tablet report, sending...
I20260808 18:27:27.272579 9475 ts_manager.cc:194] Registered new tserver with Master: 647495edaa1f493b838a4e9687d6b948 (127.7.164.65:36909)
I20260808 18:27:27.273309 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 9587
I20260808 18:27:27.273420 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:27:27.273667 7825 external_mini_cluster.cc:1428] Setting key 9ca45e0d1ac05ff417b361642b661496
I20260808 18:27:27.273886 9475 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:45493
I20260808 18:27:27.281385 9797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.272334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37137 (local address 127.7.164.65:33423)
0808 18:27:27.274410 (+ 2076us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:27.274425 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:27.274536 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:27.274795 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:27.274797 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:27.275056 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:27.275190 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.275206 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.275962 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.275965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.276519 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.276527 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.276995 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.276999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.277105 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.277638 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:27.277650 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:27.277971 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:27.279833 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:27.279836 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:27.279838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:27.280043 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:27.280353 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:27.280357 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:27.280359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:27.280447 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:27.280830 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:27.280833 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:27.280942 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 18:27:27.281050 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 18:27:27.281200 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1989,"thread_start_us":100,"threads_started":1}
I20260808 18:27:27.281373 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.272418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:33423 (local address 127.7.164.126:37137)
0808 18:27:27.272537 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.272540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.274603 (+ 2063us) server_negotiation.cc:407] Connection header received
0808 18:27:27.274641 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.274643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.274684 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.274762 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:27.275315 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.275823 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.276639 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.276866 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.276977 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.278109 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:27.278113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:27.278116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:27.278141 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:27.279707 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.280151 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.280154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.280156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.280259 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.280547 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.280550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.280552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.280729 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:27.280839 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.281108 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.281222 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:27:27.305335 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:37137
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:41817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:27.418319 9800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:27.418555 9800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:27.418596 9800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:27.422641 9800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:27.422690 9800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:27.422727 9800 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:27:27.422816 9800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:27:27.427855 9800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:41817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.164.126:37137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:27.428783 9800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:27.429744 9800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:27:27.439209 9800 server_base.cc:1034] running on GCE node
W20260808 18:27:27.439191 9809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:27.439179 9808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:27.439227 9811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:27.439917 9800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:27.440554 9800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:27.441715 9800 hybrid_clock.cc:648] HybridClock initialized: now 1786213647441694 us; error 28 us; skew 500 ppm
Aug 08 18:27:27 dist-test-slave-jztv krb5kdc[9415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:27.444494 9800 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:27:27.445297 9800 webserver.cc:460] Webserver started at http://127.7.164.66:46599/ using document root <none> and password file <none>
I20260808 18:27:27.445834 9800 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:27.445920 9800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:27.446094 9800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:27.448153 9800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "51c5740b6ca44d1182f3522a5d79bd07"
format_stamp: "Formatted at 2026-08-08 18:27:27 on dist-test-slave-jztv"
server_key: "2933e5b361bc5df2409a2156d57cdc7d"
server_key_iv: "96a2ce6c65f2df9fb1773055303d254a"
server_key_version: "encryptionkey@0"
I20260808 18:27:27.448683 9800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "51c5740b6ca44d1182f3522a5d79bd07"
format_stamp: "Formatted at 2026-08-08 18:27:27 on dist-test-slave-jztv"
server_key: "2933e5b361bc5df2409a2156d57cdc7d"
server_key_iv: "96a2ce6c65f2df9fb1773055303d254a"
server_key_version: "encryptionkey@0"
I20260808 18:27:27.452515 9800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:27:27.455001 9824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:27.456272 9800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:27.456416 9800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "51c5740b6ca44d1182f3522a5d79bd07"
format_stamp: "Formatted at 2026-08-08 18:27:27 on dist-test-slave-jztv"
server_key: "2933e5b361bc5df2409a2156d57cdc7d"
server_key_iv: "96a2ce6c65f2df9fb1773055303d254a"
server_key_version: "encryptionkey@0"
I20260808 18:27:27.456527 9800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:27.479915 9800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:27.482847 9800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:27.483052 9800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:27.484481 9800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:27.484561 9800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:27.484634 9800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:27.484683 9800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:27:27 dist-test-slave-jztv krb5kdc[9415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:27.496855 9800 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:44659
I20260808 18:27:27.496961 10006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:44659 every 8 connection(s)
I20260808 18:27:27.497972 9800 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:27:27.499660 9834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.485945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37137 (local address 127.7.164.66:54659)
0808 18:27:27.486463 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:27.486512 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:27.487488 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:27.488281 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:27.488287 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:27.488614 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:27.489067 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.489080 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.490085 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.490089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.491196 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.491206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.491862 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.491866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.492090 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.492795 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:27.492822 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:27.494637 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:27.496797 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:27.496803 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:27.496816 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:27.497155 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:27.497534 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:27.497538 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:27.497539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:27.497708 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:27.498162 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:27.498168 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:27.498507 (+ 339us) client_negotiation.cc:769] Sending connection context
0808 18:27:27.498700 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 18:27:27.498963 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":178,"threads_started":1}
I20260808 18:27:27.500144 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.485380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:54659 (local address 127.7.164.126:37137)
0808 18:27:27.485526 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.485530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.486618 (+ 1088us) server_negotiation.cc:407] Connection header received
0808 18:27:27.487750 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.487755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.487824 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.487873 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:27.489296 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.489938 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.491329 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.491700 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.491816 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.494815 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:27.494819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:27.494822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:27.494857 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:27.496620 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.497304 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.497308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.497310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.497425 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.497810 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.497812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.497813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.497991 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:27.498108 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.499879 (+ 1771us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.500000 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:27:27.501145 10008 heartbeater.cc:347] Connected to a master server at 127.7.164.126:37137
I20260808 18:27:27.501304 10008 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:27.501782 10008 heartbeater.cc:511] Master 127.7.164.126:37137 requested a full tablet report, sending...
I20260808 18:27:27.502810 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 9800
I20260808 18:27:27.502912 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:27:27.502983 9475 ts_manager.cc:194] Registered new tserver with Master: 51c5740b6ca44d1182f3522a5d79bd07 (127.7.164.66:44659)
I20260808 18:27:27.503228 7825 external_mini_cluster.cc:1428] Setting key 0319cf994b9677d86ab00b7cff56f657
I20260808 18:27:27.503724 9475 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:54659
I20260808 18:27:27.511585 10019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.503303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37137 (local address 127.7.164.66:47799)
0808 18:27:27.503615 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:27.503628 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:27.503713 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:27.504390 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:27.504393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:27.504587 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:27.504745 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.504753 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.505782 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.505788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.506800 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.506815 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.507392 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.507395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.507459 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.507960 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:27.507981 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:27.508271 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:27.509928 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:27.509935 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:27.509937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:27.510133 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:27.510489 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:27.510491 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:27.510493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:27.510574 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:27.511141 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:27.511144 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:27.511230 (+ 86us) client_negotiation.cc:769] Sending connection context
0808 18:27:27.511329 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 18:27:27.511449 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":2,"thread_start_us":132,"threads_started":1}
I20260808 18:27:27.511766 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.504075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:47799 (local address 127.7.164.126:37137)
0808 18:27:27.504180 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.504184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.504197 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:27:27.504230 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.504235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.504280 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.504369 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:27.504896 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.505611 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.506972 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.507215 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.507270 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.508415 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:27.508419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:27.508421 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:27.508445 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:27.509810 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.510270 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.510274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.510276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.510379 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.510679 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.510685 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.510690 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.511007 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:27.511129 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.511432 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.511567 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:27:27.534148 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:37137
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:41817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:27.652868 10023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:27.653116 10023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:27.653159 10023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:27.657155 10023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:27.657215 10023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:27.657253 10023 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:27:27.657322 10023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:27:27.662865 10023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:41817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.164.126:37137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:27.663942 10023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:27.664970 10023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:27:27.674957 10023 server_base.cc:1034] running on GCE node
W20260808 18:27:27.674916 10031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:27.674916 10035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:27.674966 10030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:27.675608 10023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:27.676261 10023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:27.677455 10023 hybrid_clock.cc:648] HybridClock initialized: now 1786213647677440 us; error 26 us; skew 500 ppm
Aug 08 18:27:27 dist-test-slave-jztv krb5kdc[9415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:27.680354 10023 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:27:27.681250 10023 webserver.cc:460] Webserver started at http://127.7.164.67:41437/ using document root <none> and password file <none>
I20260808 18:27:27.681826 10023 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:27.681922 10023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:27.682168 10023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:27.684235 10023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "2ece774932e8411aad70a335d13c8e65"
format_stamp: "Formatted at 2026-08-08 18:27:27 on dist-test-slave-jztv"
server_key: "a35535a0ec299c1a2af038f9c4179c12"
server_key_iv: "ca1800bd39ba7ac3650fcf3bff116fff"
server_key_version: "encryptionkey@0"
I20260808 18:27:27.684821 10023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "2ece774932e8411aad70a335d13c8e65"
format_stamp: "Formatted at 2026-08-08 18:27:27 on dist-test-slave-jztv"
server_key: "a35535a0ec299c1a2af038f9c4179c12"
server_key_iv: "ca1800bd39ba7ac3650fcf3bff116fff"
server_key_version: "encryptionkey@0"
I20260808 18:27:27.688534 10023 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:27:27.691025 10042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:27.692293 10023 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:27.692464 10023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "2ece774932e8411aad70a335d13c8e65"
format_stamp: "Formatted at 2026-08-08 18:27:27 on dist-test-slave-jztv"
server_key: "a35535a0ec299c1a2af038f9c4179c12"
server_key_iv: "ca1800bd39ba7ac3650fcf3bff116fff"
server_key_version: "encryptionkey@0"
I20260808 18:27:27.692584 10023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:27.716434 10023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:27.719707 10023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:27.719915 10023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:27.721357 10023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:27.721433 10023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:27.721516 10023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:27.721562 10023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:27:27 dist-test-slave-jztv krb5kdc[9415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213647, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:27.732820 10023 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:35587
I20260808 18:27:27.732884 10222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:35587 every 8 connection(s)
I20260808 18:27:27.733771 10023 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:27:27.735818 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.722404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:58223 (local address 127.7.164.126:37137)
0808 18:27:27.722551 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.722554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.723584 (+ 1030us) server_negotiation.cc:407] Connection header received
0808 18:27:27.724691 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.724698 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.724772 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.724875 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:27.726441 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.726975 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.728102 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.728324 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.728446 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.732138 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:27.732144 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:27.732147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:27.732175 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:27.733704 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.734256 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.734260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.734262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.734357 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.734771 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.734774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.734776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.734967 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:27.735094 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.735602 (+ 508us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.735702 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:27:27.736645 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.722937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37137 (local address 127.7.164.67:58223)
0808 18:27:27.723428 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:27.723470 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:27.724401 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:27.725074 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:27.725082 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:27.725587 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:27.726281 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.726300 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.727108 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.727112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.727958 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.727968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.728498 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.728502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.728677 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.729314 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:27.729331 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:27.731958 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:27.733828 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:27.733835 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:27.733854 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:27.734117 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:27.734465 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:27.734468 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:27.734470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:27.734642 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:27.735107 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:27.735112 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:27.735376 (+ 264us) client_negotiation.cc:769] Sending connection context
0808 18:27:27.735523 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 18:27:27.735802 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":127,"threads_started":1}
I20260808 18:27:27.737885 10223 heartbeater.cc:347] Connected to a master server at 127.7.164.126:37137
I20260808 18:27:27.738049 10223 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:27.738556 10223 heartbeater.cc:511] Master 127.7.164.126:37137 requested a full tablet report, sending...
I20260808 18:27:27.739554 9475 ts_manager.cc:194] Registered new tserver with Master: 2ece774932e8411aad70a335d13c8e65 (127.7.164.67:35587)
I20260808 18:27:27.740245 9475 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:58223
I20260808 18:27:27.741492 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 10023
I20260808 18:27:27.741632 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:27:27.741884 7825 external_mini_cluster.cc:1428] Setting key 897f1f8ac603b63000da12d3ee3db638
I20260808 18:27:27.744004 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:27:27.747493 10233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.740000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37137 (local address 127.7.164.67:49997)
0808 18:27:27.740363 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:27.740378 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:27.740475 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:27.740771 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:27.740774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:27.740966 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:27.741102 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.741109 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.742119 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.742122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.742855 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.742865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.743404 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.743407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.743500 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.743972 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:27.743986 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:27.744357 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:27.745919 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:27.745925 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:27.745930 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:27.746236 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:27.746556 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:27.746559 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:27.746561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:27.746650 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:27.747108 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:27.747110 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:27.747181 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 18:27:27.747261 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 18:27:27.747358 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260808 18:27:27.747665 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.740101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:49997 (local address 127.7.164.126:37137)
0808 18:27:27.740206 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.740212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.740470 (+ 258us) server_negotiation.cc:407] Connection header received
0808 18:27:27.740613 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.740616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.740667 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.740731 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:27.741265 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.741976 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.742985 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.743282 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.743398 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.744491 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:27.744494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:27.744496 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:27.744517 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:27.745808 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.746328 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.746331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.746333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.746424 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.746754 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.746759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.746761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.747005 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:27.747119 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.747349 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.747472 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
Aug 08 18:27:27 dist-test-slave-jztv krb5kdc[9415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213647, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:27:27 dist-test-slave-jztv krb5kdc[9415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213647, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:27.774601 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.765950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41664 (local address 127.7.164.126:37137)
0808 18:27:27.766106 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.766110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.766121 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:27:27.766157 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.766159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.766195 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.766312 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:27.767128 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.767671 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.768566 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.768833 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.768940 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.771520 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:27.771523 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:27.771525 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:27.771548 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:27.772891 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.773330 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.773335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.773336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.773419 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:27.773705 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:27.773708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:27.773710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:27.773886 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:27.774023 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.774311 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.774426 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:27:27.777526 9475 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:27:27.780633 9475 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:27:27.795120 10251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.788914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:36909 (local address 127.0.0.1:54518)
0808 18:27:27.789730 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:27.789778 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:27.789914 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:27.790765 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:27.790771 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:27.790805 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:27.791030 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.791041 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.792456 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.792460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.793724 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.793732 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.794577 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.794581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.794687 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.794723 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 18:27:27.794832 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 18:27:27.794956 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"mutex_wait_us":154,"spinlock_wait_cycles":11136,"thread_start_us":58,"threads_started":1}
I20260808 18:27:27.795276 10254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.789528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.7.164.66:44659)
0808 18:27:27.790397 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.790408 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.790428 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:27:27.790492 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.790506 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.790666 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.791005 (+ 339us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:27.791366 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.792463 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.793854 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.794667 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.794809 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.794958 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.795021 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.795124 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":765,"thread_start_us":120,"threads_started":1}
I20260808 18:27:27.795303 10248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.789369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:35587 (local address 127.0.0.1:41334)
0808 18:27:27.789904 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:27.789917 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:27.790005 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:27.790765 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:27.790771 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:27.790804 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:27.791081 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.791088 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.792143 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.792147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.793534 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.793545 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.794868 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.794872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.794998 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.795021 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:27:27.795095 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 18:27:27.795159 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":10,"spinlock_wait_cycles":5120,"thread_start_us":94,"threads_started":1}
I20260808 18:27:27.795172 10252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.789107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:44659 (local address 127.0.0.1:57382)
0808 18:27:27.789731 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:27.789783 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:27.789913 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:27.790818 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:27.790821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:27.790838 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:27.791237 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.791245 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.792585 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.792588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.793715 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.793726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.794814 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.794820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.794914 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.794936 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 18:27:27.795006 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 18:27:27.795059 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":76,"spinlock_wait_cycles":7040,"thread_start_us":96,"threads_started":1}
I20260808 18:27:27.795759 10249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.789060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54518 (local address 127.7.164.65:36909)
0808 18:27:27.790250 (+ 1190us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.790260 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.790283 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:27:27.790362 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.790367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.790597 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.790874 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:27.791162 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.792339 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.793835 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.794448 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.794576 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.794726 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.795492 (+ 766us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.795607 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1039,"thread_start_us":100,"threads_started":1}
I20260808 18:27:27.795826 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.789693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41334 (local address 127.7.164.67:35587)
0808 18:27:27.790249 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.790263 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.790287 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:27:27.790364 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.790369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.790569 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.790725 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:27.791238 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.792024 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.793674 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.794522 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.794637 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.794806 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.795577 (+ 771us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.795654 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":111,"threads_started":1}
I20260808 18:27:27.798576 10125 tablet_service.cc:1467] Processing CreateTablet for tablet aa257fe7ad334cb6a109c273a3ab519b (DEFAULT_TABLE table=test-table [id=56dc1e35bda6487fb54c6b46e675fcc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:27.798593 9907 tablet_service.cc:1467] Processing CreateTablet for tablet aa257fe7ad334cb6a109c273a3ab519b (DEFAULT_TABLE table=test-table [id=56dc1e35bda6487fb54c6b46e675fcc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:27.798591 9697 tablet_service.cc:1467] Processing CreateTablet for tablet aa257fe7ad334cb6a109c273a3ab519b (DEFAULT_TABLE table=test-table [id=56dc1e35bda6487fb54c6b46e675fcc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:27.799578 10125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa257fe7ad334cb6a109c273a3ab519b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:27.799613 9907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa257fe7ad334cb6a109c273a3ab519b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:27.799613 9697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa257fe7ad334cb6a109c273a3ab519b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:27.806636 10256 tablet_bootstrap.cc:492] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65: Bootstrap starting.
I20260808 18:27:27.807927 10258 tablet_bootstrap.cc:492] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948: Bootstrap starting.
I20260808 18:27:27.808162 10260 tablet_bootstrap.cc:492] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07: Bootstrap starting.
I20260808 18:27:27.808876 10256 tablet_bootstrap.cc:654] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:27.809612 10256 log.cc:824] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:27.810083 10258 tablet_bootstrap.cc:654] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:27.810083 10260 tablet_bootstrap.cc:654] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:27.810926 10258 log.cc:824] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:27.811197 10260 log.cc:824] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:27.811769 10256 tablet_bootstrap.cc:492] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65: No bootstrap required, opened a new log
I20260808 18:27:27.811985 10256 ts_tablet_manager.cc:1397] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:27:27.812986 10258 tablet_bootstrap.cc:492] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948: No bootstrap required, opened a new log
I20260808 18:27:27.813110 10260 tablet_bootstrap.cc:492] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07: No bootstrap required, opened a new log
I20260808 18:27:27.813222 10258 ts_tablet_manager.cc:1397] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 18:27:27.813292 10260 ts_tablet_manager.cc:1397] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:27:27.816069 10256 raft_consensus.cc:348] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.816349 10256 raft_consensus.cc:374] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:27.816411 10256 raft_consensus.cc:729] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ece774932e8411aad70a335d13c8e65, State: Initialized, Role: FOLLOWER
I20260808 18:27:27.816823 10258 raft_consensus.cc:348] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.817081 10258 raft_consensus.cc:374] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:27.816990 10256 consensus_queue.cc:260] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.817135 10258 raft_consensus.cc:729] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647495edaa1f493b838a4e9687d6b948, State: Initialized, Role: FOLLOWER
I20260808 18:27:27.817569 10258 consensus_queue.cc:260] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.817777 10223 heartbeater.cc:503] Master 127.7.164.126:37137 was elected leader, sending a full tablet report...
I20260808 18:27:27.818164 10256 ts_tablet_manager.cc:1428] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 18:27:27.817072 10260 raft_consensus.cc:348] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.818254 10260 raft_consensus.cc:374] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:27.818310 10260 raft_consensus.cc:729] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c5740b6ca44d1182f3522a5d79bd07, State: Initialized, Role: FOLLOWER
I20260808 18:27:27.818521 10258 ts_tablet_manager.cc:1428] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260808 18:27:27.818832 9788 heartbeater.cc:503] Master 127.7.164.126:37137 was elected leader, sending a full tablet report...
I20260808 18:27:27.818769 10260 consensus_queue.cc:260] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.819520 10008 heartbeater.cc:503] Master 127.7.164.126:37137 was elected leader, sending a full tablet report...
I20260808 18:27:27.819716 10260 ts_tablet_manager.cc:1428] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:27:27.848057 10264 raft_consensus.cc:482] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:27:27.848292 10264 raft_consensus.cc:504] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.849725 10264 leader_election.cc:288] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647495edaa1f493b838a4e9687d6b948 (127.7.164.65:36909), 51c5740b6ca44d1182f3522a5d79bd07 (127.7.164.66:44659)
I20260808 18:27:27.853736 10254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.850116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:57665 (local address 127.7.164.66:44659)
0808 18:27:27.850276 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.850280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.850522 (+ 242us) server_negotiation.cc:407] Connection header received
0808 18:27:27.850684 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.850687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.850738 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.850824 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:27.851105 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.851604 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.852744 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.853161 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.853291 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.853348 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.853545 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.853613 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:27:27.853753 10269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.849978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:44659 (local address 127.7.164.67:57665)
0808 18:27:27.850418 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:27.850429 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:27.850566 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:27.850837 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:27.850839 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:27.850855 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:27.850992 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.850998 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.851719 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.851725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.852616 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.852626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.853289 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.853293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.853398 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.853421 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:27:27.853530 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 18:27:27.853603 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":34,"thread_start_us":124,"threads_started":1}
I20260808 18:27:27.853891 10249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.850088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:59419 (local address 127.7.164.65:36909)
0808 18:27:27.850242 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.850249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.850267 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:27:27.850449 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.850451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.850505 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.850590 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:27.850876 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.851381 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.852910 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.853330 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.853460 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.853521 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.853699 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.853768 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:27:27.853909 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.849960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:36909 (local address 127.7.164.67:59419)
0808 18:27:27.850168 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:27.850184 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:27.850310 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:27.850615 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:27.850619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:27.850637 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:27.850763 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.850769 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.851491 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.851496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.852778 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.852788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.853452 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.853455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.853562 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.853591 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 18:27:27.853681 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 18:27:27.853752 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":58}
I20260808 18:27:27.854651 9939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa257fe7ad334cb6a109c273a3ab519b" candidate_uuid: "2ece774932e8411aad70a335d13c8e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c5740b6ca44d1182f3522a5d79bd07" is_pre_election: true
I20260808 18:27:27.854667 9725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa257fe7ad334cb6a109c273a3ab519b" candidate_uuid: "2ece774932e8411aad70a335d13c8e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647495edaa1f493b838a4e9687d6b948" is_pre_election: true
I20260808 18:27:27.854946 9725 raft_consensus.cc:2454] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ece774932e8411aad70a335d13c8e65 in term 0.
I20260808 18:27:27.854954 9939 raft_consensus.cc:2454] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ece774932e8411aad70a335d13c8e65 in term 0.
I20260808 18:27:27.855425 10050 leader_election.cc:302] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ece774932e8411aad70a335d13c8e65, 51c5740b6ca44d1182f3522a5d79bd07; no voters:
I20260808 18:27:27.855710 10264 raft_consensus.cc:2781] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:27:27.855804 10264 raft_consensus.cc:482] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:27:27.855887 10264 raft_consensus.cc:3037] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:27.856948 10264 raft_consensus.cc:504] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.857332 10264 leader_election.cc:288] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [CANDIDATE]: Term 1 election: Requested vote from peers 647495edaa1f493b838a4e9687d6b948 (127.7.164.65:36909), 51c5740b6ca44d1182f3522a5d79bd07 (127.7.164.66:44659)
I20260808 18:27:27.857667 9939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa257fe7ad334cb6a109c273a3ab519b" candidate_uuid: "2ece774932e8411aad70a335d13c8e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c5740b6ca44d1182f3522a5d79bd07"
I20260808 18:27:27.857748 9725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa257fe7ad334cb6a109c273a3ab519b" candidate_uuid: "2ece774932e8411aad70a335d13c8e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647495edaa1f493b838a4e9687d6b948"
I20260808 18:27:27.857807 9939 raft_consensus.cc:3037] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:27.857903 9725 raft_consensus.cc:3037] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:27.859040 9725 raft_consensus.cc:2454] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ece774932e8411aad70a335d13c8e65 in term 1.
I20260808 18:27:27.859323 9939 raft_consensus.cc:2454] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ece774932e8411aad70a335d13c8e65 in term 1.
I20260808 18:27:27.859361 10045 leader_election.cc:302] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ece774932e8411aad70a335d13c8e65, 647495edaa1f493b838a4e9687d6b948; no voters:
I20260808 18:27:27.859659 10264 raft_consensus.cc:2781] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:27.859929 10264 raft_consensus.cc:686] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [term 1 LEADER]: Becoming Leader. State: Replica: 2ece774932e8411aad70a335d13c8e65, State: Running, Role: LEADER
I20260808 18:27:27.860265 10264 consensus_queue.cc:237] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.863511 9473 catalog_manager.cc:5495] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ece774932e8411aad70a335d13c8e65 (127.7.164.67). New cstate: current_term: 1 leader_uuid: "2ece774932e8411aad70a335d13c8e65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } health_report { overall_health: HEALTHY } } }
I20260808 18:27:27.897657 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.893755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41348 (local address 127.7.164.67:35587)
0808 18:27:27.893884 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.893888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.893909 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:27:27.894110 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.894112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.894165 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.894327 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:27.894614 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.895127 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.896112 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.896330 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.896445 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.896786 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:27.897231 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:27.897351 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.897459 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.897510 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:27:27.903424 9725 raft_consensus.cc:1261] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948 [term 1 FOLLOWER]: Refusing update from remote peer 2ece774932e8411aad70a335d13c8e65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:27.903415 9939 raft_consensus.cc:1261] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07 [term 1 FOLLOWER]: Refusing update from remote peer 2ece774932e8411aad70a335d13c8e65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:27.904112 10270 consensus_queue.cc:1035] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:27.904312 10264 consensus_queue.cc:1035] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:27.924157 9476 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:27.924346 9476 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:27.926903 9476 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9: Sending DeleteTablet for 3 replicas of tablet aa257fe7ad334cb6a109c273a3ab519b
I20260808 18:27:27.927703 9907 tablet_service.cc:1514] Processing DeleteTablet for tablet aa257fe7ad334cb6a109c273a3ab519b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:27 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:57382
I20260808 18:27:27.927846 10125 tablet_service.cc:1514] Processing DeleteTablet for tablet aa257fe7ad334cb6a109c273a3ab519b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:27 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:41334
I20260808 18:27:27.928041 9697 tablet_service.cc:1514] Processing DeleteTablet for tablet aa257fe7ad334cb6a109c273a3ab519b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:27 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:54518
I20260808 18:27:27.928221 10286 tablet_replica.cc:330] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07: stopping tablet replica
I20260808 18:27:27.928488 10289 tablet_replica.cc:330] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948: stopping tablet replica
I20260808 18:27:27.928571 10286 raft_consensus.cc:2229] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:27.928750 10289 raft_consensus.cc:2229] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:27.928880 10286 raft_consensus.cc:2258] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:27.928993 10289 raft_consensus.cc:2258] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:27.929165 9476 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:27:27.929574 9476 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:27:27.930395 10286 ts_tablet_manager.cc:1905] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:27.930395 10289 ts_tablet_manager.cc:1905] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:27.931481 10288 tablet_replica.cc:330] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65: stopping tablet replica
I20260808 18:27:27.931802 10288 raft_consensus.cc:2229] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:27:27.932219 10288 raft_consensus.cc:2258] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:27.933166 10286 ts_tablet_manager.cc:1918] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:27:27.933261 10286 log.cc:1196] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/aa257fe7ad334cb6a109c273a3ab519b
I20260808 18:27:27.933625 10286 ts_tablet_manager.cc:1939] T aa257fe7ad334cb6a109c273a3ab519b P 51c5740b6ca44d1182f3522a5d79bd07: Deleting consensus metadata
I20260808 18:27:27.934583 10288 ts_tablet_manager.cc:1905] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:27.935073 9458 catalog_manager.cc:4841] TS 51c5740b6ca44d1182f3522a5d79bd07 (127.7.164.66:44659): tablet aa257fe7ad334cb6a109c273a3ab519b (table test-table [id=56dc1e35bda6487fb54c6b46e675fcc2]) successfully deleted
I20260808 18:27:27.935456 9697 tablet_service.cc:1467] Processing CreateTablet for tablet ac83c0df93f0462eb78bfda16fcc9c93 (DEFAULT_TABLE table=test-table [id=2634e39d70c94cbd93c832071b7999a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:27.935540 9907 tablet_service.cc:1467] Processing CreateTablet for tablet ac83c0df93f0462eb78bfda16fcc9c93 (DEFAULT_TABLE table=test-table [id=2634e39d70c94cbd93c832071b7999a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:27.935727 9697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac83c0df93f0462eb78bfda16fcc9c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:27.935752 9907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac83c0df93f0462eb78bfda16fcc9c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:27.937924 10260 tablet_bootstrap.cc:492] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07: Bootstrap starting.
I20260808 18:27:27.938103 10125 tablet_service.cc:1467] Processing CreateTablet for tablet ac83c0df93f0462eb78bfda16fcc9c93 (DEFAULT_TABLE table=test-table [id=2634e39d70c94cbd93c832071b7999a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:27.938373 10125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac83c0df93f0462eb78bfda16fcc9c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:27.938968 10260 tablet_bootstrap.cc:654] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:27.939286 10288 ts_tablet_manager.cc:1918] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:27:27.939465 10288 log.cc:1196] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/aa257fe7ad334cb6a109c273a3ab519b
I20260808 18:27:27.939858 10288 ts_tablet_manager.cc:1939] T aa257fe7ad334cb6a109c273a3ab519b P 2ece774932e8411aad70a335d13c8e65: Deleting consensus metadata
I20260808 18:27:27.940086 10289 ts_tablet_manager.cc:1918] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:27:27.940187 10289 log.cc:1196] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/wal/wals/aa257fe7ad334cb6a109c273a3ab519b
I20260808 18:27:27.940534 10289 ts_tablet_manager.cc:1939] T aa257fe7ad334cb6a109c273a3ab519b P 647495edaa1f493b838a4e9687d6b948: Deleting consensus metadata
I20260808 18:27:27.940753 10256 tablet_bootstrap.cc:492] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65: Bootstrap starting.
I20260808 18:27:27.940842 10258 tablet_bootstrap.cc:492] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948: Bootstrap starting.
I20260808 18:27:27.941644 10256 tablet_bootstrap.cc:654] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:27.941761 10258 tablet_bootstrap.cc:654] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:27.942379 9453 catalog_manager.cc:4841] TS 647495edaa1f493b838a4e9687d6b948 (127.7.164.65:36909): tablet aa257fe7ad334cb6a109c273a3ab519b (table test-table [id=56dc1e35bda6487fb54c6b46e675fcc2]) successfully deleted
I20260808 18:27:27.942620 10260 tablet_bootstrap.cc:492] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07: No bootstrap required, opened a new log
I20260808 18:27:27.942713 10260 ts_tablet_manager.cc:1397] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 18:27:27.942998 10258 tablet_bootstrap.cc:492] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948: No bootstrap required, opened a new log
I20260808 18:27:27.943082 10258 ts_tablet_manager.cc:1397] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:27.943219 10256 tablet_bootstrap.cc:492] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65: No bootstrap required, opened a new log
I20260808 18:27:27.943205 10260 raft_consensus.cc:348] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.943295 10256 ts_tablet_manager.cc:1397] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 18:27:27.943320 10260 raft_consensus.cc:374] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:27.943360 10260 raft_consensus.cc:729] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c5740b6ca44d1182f3522a5d79bd07, State: Initialized, Role: FOLLOWER
I20260808 18:27:27.943482 10260 consensus_queue.cc:260] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.943514 10258 raft_consensus.cc:348] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.943615 10258 raft_consensus.cc:374] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:27.943655 10258 raft_consensus.cc:729] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647495edaa1f493b838a4e9687d6b948, State: Initialized, Role: FOLLOWER
I20260808 18:27:27.943759 10260 ts_tablet_manager.cc:1428] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:27.943722 10256 raft_consensus.cc:348] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.943818 10256 raft_consensus.cc:374] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:27.943781 10258 consensus_queue.cc:260] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.943856 10256 raft_consensus.cc:729] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ece774932e8411aad70a335d13c8e65, State: Initialized, Role: FOLLOWER
I20260808 18:27:27.943988 10256 consensus_queue.cc:260] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.944056 10258 ts_tablet_manager.cc:1428] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:27.944253 10256 ts_tablet_manager.cc:1428] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:27:27.945024 9452 catalog_manager.cc:4841] TS 2ece774932e8411aad70a335d13c8e65 (127.7.164.67:35587): tablet aa257fe7ad334cb6a109c273a3ab519b (table test-table [id=56dc1e35bda6487fb54c6b46e675fcc2]) successfully deleted
I20260808 18:27:27.975226 10265 raft_consensus.cc:482] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:27:27.975387 10265 raft_consensus.cc:504] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.976675 10265 leader_election.cc:288] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51c5740b6ca44d1182f3522a5d79bd07 (127.7.164.66:44659), 2ece774932e8411aad70a335d13c8e65 (127.7.164.67:35587)
I20260808 18:27:27.981653 10254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.977376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:43675 (local address 127.7.164.66:44659)
0808 18:27:27.977501 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.977506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.977625 (+ 119us) server_negotiation.cc:407] Connection header received
0808 18:27:27.977843 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.977850 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.977905 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.977951 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:27.978417 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.979212 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.980412 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.981024 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.981146 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.981201 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.981454 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.981528 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:27:27.981673 10298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.976964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:44659 (local address 127.7.164.65:43675)
0808 18:27:27.977523 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:27.977537 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:27.977704 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:27.978106 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:27.978109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:27.978144 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:27.978294 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.978301 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.979382 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.979385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.980284 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.980294 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.981194 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.981200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.981306 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.981335 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 18:27:27.981421 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 18:27:27.981492 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"spinlock_wait_cycles":3584,"thread_start_us":106,"threads_started":1}
I20260808 18:27:27.982487 9939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac83c0df93f0462eb78bfda16fcc9c93" candidate_uuid: "647495edaa1f493b838a4e9687d6b948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c5740b6ca44d1182f3522a5d79bd07" is_pre_election: true
I20260808 18:27:27.982595 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.977589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:41957 (local address 127.7.164.67:35587)
0808 18:27:27.977700 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:27.977704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:27.977786 (+ 82us) server_negotiation.cc:407] Connection header received
0808 18:27:27.977900 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:27.977903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:27.977950 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:27.978036 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:27.979386 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.980163 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.981241 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.981849 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.981957 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.982021 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:27.982399 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 18:27:27.982480 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:27:27.982604 10296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:27.976972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:35587 (local address 127.7.164.65:41957)
0808 18:27:27.977691 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:27.977704 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:27.977794 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:27.978074 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:27.978077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:27.978099 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:27.978296 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.978301 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.980293 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.980296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.981062 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:27.981073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:27.982115 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:27.982118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:27.982264 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:27.982290 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 18:27:27.982381 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 18:27:27.982448 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":47,"spinlock_wait_cycles":768,"thread_start_us":110,"threads_started":1}
I20260808 18:27:27.982653 9939 raft_consensus.cc:2454] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647495edaa1f493b838a4e9687d6b948 in term 0.
I20260808 18:27:27.983059 9616 leader_election.cc:302] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c5740b6ca44d1182f3522a5d79bd07, 647495edaa1f493b838a4e9687d6b948; no voters:
I20260808 18:27:27.983258 10155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac83c0df93f0462eb78bfda16fcc9c93" candidate_uuid: "647495edaa1f493b838a4e9687d6b948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ece774932e8411aad70a335d13c8e65" is_pre_election: true
I20260808 18:27:27.983355 10265 raft_consensus.cc:2781] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:27:27.983418 10265 raft_consensus.cc:482] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:27:27.983464 10265 raft_consensus.cc:3037] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:27.983489 10155 raft_consensus.cc:2454] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647495edaa1f493b838a4e9687d6b948 in term 0.
I20260808 18:27:27.984579 10265 raft_consensus.cc:504] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
W20260808 18:27:27.984771 10225 tablet.cc:2369] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:27:27.985001 10265 leader_election.cc:288] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [CANDIDATE]: Term 1 election: Requested vote from peers 51c5740b6ca44d1182f3522a5d79bd07 (127.7.164.66:44659), 2ece774932e8411aad70a335d13c8e65 (127.7.164.67:35587)
I20260808 18:27:27.985314 9939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac83c0df93f0462eb78bfda16fcc9c93" candidate_uuid: "647495edaa1f493b838a4e9687d6b948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51c5740b6ca44d1182f3522a5d79bd07"
I20260808 18:27:27.985448 9939 raft_consensus.cc:3037] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:27.985579 10155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac83c0df93f0462eb78bfda16fcc9c93" candidate_uuid: "647495edaa1f493b838a4e9687d6b948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ece774932e8411aad70a335d13c8e65"
I20260808 18:27:27.985682 10155 raft_consensus.cc:3037] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:27.986480 9939 raft_consensus.cc:2454] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647495edaa1f493b838a4e9687d6b948 in term 1.
I20260808 18:27:27.986760 9616 leader_election.cc:302] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c5740b6ca44d1182f3522a5d79bd07, 647495edaa1f493b838a4e9687d6b948; no voters:
I20260808 18:27:27.986970 10265 raft_consensus.cc:2781] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:27.987191 10265 raft_consensus.cc:686] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [term 1 LEADER]: Becoming Leader. State: Replica: 647495edaa1f493b838a4e9687d6b948, State: Running, Role: LEADER
I20260808 18:27:27.987394 10155 raft_consensus.cc:2454] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647495edaa1f493b838a4e9687d6b948 in term 1.
I20260808 18:27:27.987462 10265 consensus_queue.cc:237] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:27.989919 9476 catalog_manager.cc:5495] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 reported cstate change: term changed from 0 to 1, leader changed from <none> to 647495edaa1f493b838a4e9687d6b948 (127.7.164.65). New cstate: current_term: 1 leader_uuid: "647495edaa1f493b838a4e9687d6b948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } health_report { overall_health: UNKNOWN } } }
W20260808 18:27:27.999197 10011 tablet.cc:2369] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:27:28.017144 9789 tablet.cc:2369] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 18:27:28 dist-test-slave-jztv krb5kdc[9415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213646, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:28.033171 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.023955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41670 (local address 127.7.164.126:37137)
0808 18:27:28.024105 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:28.024110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:28.024239 (+ 129us) server_negotiation.cc:407] Connection header received
0808 18:27:28.024358 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:28.024361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:28.024411 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:28.024509 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:28.025129 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.025889 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.026791 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.027133 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.027240 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.029764 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:28.029767 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:28.029769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:28.029791 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:28.031254 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:28.031849 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:28.031852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:28.031853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:28.031935 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:28.032257 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:28.032260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:28.032261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:28.032445 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:28.032523 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:28.032912 (+ 389us) server_negotiation.cc:299] Negotiation successful
0808 18:27:28.033025 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":53}
I20260808 18:27:28.033197 10313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.023846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37137 (local address 127.0.0.1:41670)
0808 18:27:28.024141 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:28.024156 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:28.024269 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:28.024539 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:28.024541 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:28.024903 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:28.025004 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:28.025010 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.026009 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.026012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:28.026672 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:28.026681 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.027254 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.027257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:28.027420 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.028055 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:28.028067 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:28.029611 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:28.031374 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:28.031385 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:28.031392 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:28.031750 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:28.032030 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:28.032034 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:28.032038 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:28.032143 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:28.032555 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:28.032564 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:28.032690 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 18:27:28.032816 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 18:27:28.032965 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":29,"thread_start_us":118,"threads_started":1}
I20260808 18:27:28.037523 9476 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:41670:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 18:27:28.038378 9476 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:27:28.043820 9907 tablet_service.cc:1467] Processing CreateTablet for tablet 39f524d25f344a6f8e5421d7fec4b31c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7f43b2263254235b02ca3556609dc8b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:27:28.043829 9697 tablet_service.cc:1467] Processing CreateTablet for tablet 39f524d25f344a6f8e5421d7fec4b31c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7f43b2263254235b02ca3556609dc8b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:27:28.044253 9697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39f524d25f344a6f8e5421d7fec4b31c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:28.044257 9907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39f524d25f344a6f8e5421d7fec4b31c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:28.046826 10260 tablet_bootstrap.cc:492] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07: Bootstrap starting.
I20260808 18:27:28.047324 10125 tablet_service.cc:1467] Processing CreateTablet for tablet 39f524d25f344a6f8e5421d7fec4b31c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7f43b2263254235b02ca3556609dc8b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:27:28.047714 10125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39f524d25f344a6f8e5421d7fec4b31c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:28.047726 10260 tablet_bootstrap.cc:654] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:28.048995 10260 tablet_bootstrap.cc:492] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07: No bootstrap required, opened a new log
I20260808 18:27:28.049101 10260 ts_tablet_manager.cc:1397] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:28.049518 10260 raft_consensus.cc:348] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:28.049640 10260 raft_consensus.cc:374] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:28.049681 10260 raft_consensus.cc:729] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c5740b6ca44d1182f3522a5d79bd07, State: Initialized, Role: FOLLOWER
I20260808 18:27:28.049809 10260 consensus_queue.cc:260] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:28.050091 10260 ts_tablet_manager.cc:1428] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:28.050333 10267 tablet_replica.cc:439] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } } }
I20260808 18:27:28.050469 10267 tablet_replica.cc:442] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:27:28.051390 10258 tablet_bootstrap.cc:492] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948: Bootstrap starting.
I20260808 18:27:28.051620 10256 tablet_bootstrap.cc:492] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65: Bootstrap starting.
I20260808 18:27:28.052310 10258 tablet_bootstrap.cc:654] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:28.052611 10256 tablet_bootstrap.cc:654] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:28.053472 10258 tablet_bootstrap.cc:492] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948: No bootstrap required, opened a new log
I20260808 18:27:28.053580 10258 ts_tablet_manager.cc:1397] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:28.054065 10258 raft_consensus.cc:348] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:28.054183 10258 raft_consensus.cc:374] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:28.054256 10258 raft_consensus.cc:729] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647495edaa1f493b838a4e9687d6b948, State: Initialized, Role: FOLLOWER
I20260808 18:27:28.054405 10258 consensus_queue.cc:260] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:28.054679 10300 tablet_replica.cc:439] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } } }
I20260808 18:27:28.054805 10300 tablet_replica.cc:442] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:27:28.054857 10256 tablet_bootstrap.cc:492] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65: No bootstrap required, opened a new log
I20260808 18:27:28.054936 10256 ts_tablet_manager.cc:1397] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:27:28.055049 10258 ts_tablet_manager.cc:1428] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:28.055366 10256 raft_consensus.cc:348] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:28.055490 10256 raft_consensus.cc:374] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:28.055533 10256 raft_consensus.cc:729] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ece774932e8411aad70a335d13c8e65, State: Initialized, Role: FOLLOWER
I20260808 18:27:28.055660 10256 consensus_queue.cc:260] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:28.055919 10256 ts_tablet_manager.cc:1428] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:27:28.055900 10270 tablet_replica.cc:439] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } } }
I20260808 18:27:28.056042 10270 tablet_replica.cc:442] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:27:28.334043 10267 raft_consensus.cc:482] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:27:28.334280 10267 raft_consensus.cc:504] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:28.335569 10267 leader_election.cc:288] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647495edaa1f493b838a4e9687d6b948 (127.7.164.65:36909), 2ece774932e8411aad70a335d13c8e65 (127.7.164.67:35587)
I20260808 18:27:28.340034 10249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.335943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:34353 (local address 127.7.164.65:36909)
0808 18:27:28.336092 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:28.336097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:28.336232 (+ 135us) server_negotiation.cc:407] Connection header received
0808 18:27:28.336498 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:28.336503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:28.336554 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:28.336603 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:28.337109 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.337667 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.338990 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.339440 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.339577 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.339633 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:28.339841 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 18:27:28.339919 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 18:27:28.340063 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.335758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:36909 (local address 127.7.164.66:34353)
0808 18:27:28.336140 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:28.336155 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:28.336324 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:28.336755 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:28.336758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:28.336801 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:28.336948 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:28.336956 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.337813 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.337816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:28.338853 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:28.338863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.339588 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.339591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:28.339703 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.339722 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 18:27:28.339808 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 18:27:28.339883 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":64,"spinlock_wait_cycles":384,"thread_start_us":116,"threads_started":1}
I20260808 18:27:28.340206 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.336202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:52771 (local address 127.7.164.67:35587)
0808 18:27:28.336324 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:28.336329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:28.336343 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:27:28.336567 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:28.336570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:28.336613 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:28.336705 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:28.337161 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.337692 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.339188 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.339635 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.339762 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.339811 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:28.339986 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 18:27:28.340080 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:27:28.340230 10320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.335914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:35587 (local address 127.7.164.66:52771)
0808 18:27:28.336258 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:28.336273 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:28.336386 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:28.336718 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:28.336721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:28.336738 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:28.337050 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:28.337060 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.337809 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.337815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:28.339069 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:28.339079 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.339754 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.339758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:28.339860 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.339880 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 18:27:28.339968 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 18:27:28.340082 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":35,"spinlock_wait_cycles":17152,"thread_start_us":82,"threads_started":1}
I20260808 18:27:28.340507 9725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39f524d25f344a6f8e5421d7fec4b31c" candidate_uuid: "51c5740b6ca44d1182f3522a5d79bd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647495edaa1f493b838a4e9687d6b948" is_pre_election: true
I20260808 18:27:28.340591 10155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39f524d25f344a6f8e5421d7fec4b31c" candidate_uuid: "51c5740b6ca44d1182f3522a5d79bd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ece774932e8411aad70a335d13c8e65" is_pre_election: true
I20260808 18:27:28.340719 9725 raft_consensus.cc:2454] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51c5740b6ca44d1182f3522a5d79bd07 in term 0.
I20260808 18:27:28.340739 10155 raft_consensus.cc:2454] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51c5740b6ca44d1182f3522a5d79bd07 in term 0.
I20260808 18:27:28.341178 9826 leader_election.cc:302] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2ece774932e8411aad70a335d13c8e65, 51c5740b6ca44d1182f3522a5d79bd07, 647495edaa1f493b838a4e9687d6b948; no voters:
I20260808 18:27:28.341454 10267 raft_consensus.cc:2781] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:27:28.341532 10267 raft_consensus.cc:482] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:27:28.341588 10267 raft_consensus.cc:3037] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:28.342885 10267 raft_consensus.cc:504] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:28.343237 10267 leader_election.cc:288] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [CANDIDATE]: Term 1 election: Requested vote from peers 647495edaa1f493b838a4e9687d6b948 (127.7.164.65:36909), 2ece774932e8411aad70a335d13c8e65 (127.7.164.67:35587)
I20260808 18:27:28.343627 10155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39f524d25f344a6f8e5421d7fec4b31c" candidate_uuid: "51c5740b6ca44d1182f3522a5d79bd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ece774932e8411aad70a335d13c8e65"
I20260808 18:27:28.343627 9725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39f524d25f344a6f8e5421d7fec4b31c" candidate_uuid: "51c5740b6ca44d1182f3522a5d79bd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647495edaa1f493b838a4e9687d6b948"
I20260808 18:27:28.343748 10155 raft_consensus.cc:3037] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:28.343756 9725 raft_consensus.cc:3037] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:28.344873 9725 raft_consensus.cc:2454] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c5740b6ca44d1182f3522a5d79bd07 in term 1.
I20260808 18:27:28.344880 10155 raft_consensus.cc:2454] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c5740b6ca44d1182f3522a5d79bd07 in term 1.
I20260808 18:27:28.345180 9827 leader_election.cc:302] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51c5740b6ca44d1182f3522a5d79bd07, 647495edaa1f493b838a4e9687d6b948; no voters:
I20260808 18:27:28.345336 10267 raft_consensus.cc:2781] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:28.345521 10267 raft_consensus.cc:686] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [term 1 LEADER]: Becoming Leader. State: Replica: 51c5740b6ca44d1182f3522a5d79bd07, State: Running, Role: LEADER
I20260808 18:27:28.345770 10267 consensus_queue.cc:237] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } }
I20260808 18:27:28.347304 10322 tablet_replica.cc:439] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07: TxnStatusTablet state changed. Reason: New leader 51c5740b6ca44d1182f3522a5d79bd07. Latest consensus state: current_term: 1 leader_uuid: "51c5740b6ca44d1182f3522a5d79bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } } }
I20260808 18:27:28.347427 10322 tablet_replica.cc:442] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:27:28.347685 10326 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 18:27:28.347851 10326 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 18:27:28.348583 9476 catalog_manager.cc:5495] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51c5740b6ca44d1182f3522a5d79bd07 (127.7.164.66). New cstate: current_term: 1 leader_uuid: "51c5740b6ca44d1182f3522a5d79bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } health_report { overall_health: UNKNOWN } } }
I20260808 18:27:28.417419 10254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.413601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.7.164.66:44659)
0808 18:27:28.413749 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:28.413753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:28.413769 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:27:28.413959 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:28.413962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:28.414016 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:28.414123 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:28.414402 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.414984 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.415860 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.416084 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.416238 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.416547 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:28.416983 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:28.417127 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:28.417213 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 18:27:28.417268 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 18:27:28.417424 10313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.413497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:44659 (local address 127.0.0.1:57398)
0808 18:27:28.413638 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:28.413657 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:28.413768 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:28.414134 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:28.414137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:28.414144 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:27:28.414283 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:28.414291 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.415101 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.415106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:28.415738 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:28.415747 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.416238 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.416242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:28.416340 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.416386 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:27:28.417110 (+ 724us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:27:28.417128 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 18:27:28.417215 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 18:27:28.417298 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 18:27:28.421066 9725 raft_consensus.cc:1261] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948 [term 1 FOLLOWER]: Refusing update from remote peer 51c5740b6ca44d1182f3522a5d79bd07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:28.421176 10155 raft_consensus.cc:1261] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65 [term 1 FOLLOWER]: Refusing update from remote peer 51c5740b6ca44d1182f3522a5d79bd07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:28.421666 10267 consensus_queue.cc:1035] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:28.421849 10322 consensus_queue.cc:1035] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:28.423295 10300 tablet_replica.cc:439] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948: TxnStatusTablet state changed. Reason: New leader 51c5740b6ca44d1182f3522a5d79bd07. Latest consensus state: current_term: 1 leader_uuid: "51c5740b6ca44d1182f3522a5d79bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } } }
I20260808 18:27:28.423424 10300 tablet_replica.cc:442] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:27:28.423625 10270 tablet_replica.cc:439] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65: TxnStatusTablet state changed. Reason: New leader 51c5740b6ca44d1182f3522a5d79bd07. Latest consensus state: current_term: 1 leader_uuid: "51c5740b6ca44d1182f3522a5d79bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } } }
I20260808 18:27:28.423736 10270 tablet_replica.cc:442] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:27:28.426255 10322 tablet_replica.cc:439] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "51c5740b6ca44d1182f3522a5d79bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } } }
I20260808 18:27:28.426378 10322 tablet_replica.cc:442] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:27:28.426487 10322 tablet_replica.cc:439] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "51c5740b6ca44d1182f3522a5d79bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } } }
I20260808 18:27:28.426546 10322 tablet_replica.cc:442] T 39f524d25f344a6f8e5421d7fec4b31c P 51c5740b6ca44d1182f3522a5d79bd07: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:27:28.427802 10300 tablet_replica.cc:439] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "51c5740b6ca44d1182f3522a5d79bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } } }
I20260808 18:27:28.427939 10300 tablet_replica.cc:442] T 39f524d25f344a6f8e5421d7fec4b31c P 647495edaa1f493b838a4e9687d6b948: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:27:28.428575 10270 tablet_replica.cc:439] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "51c5740b6ca44d1182f3522a5d79bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647495edaa1f493b838a4e9687d6b948" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 36909 } } peers { permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 } } peers { permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 } } }
I20260808 18:27:28.428679 10270 tablet_replica.cc:442] T 39f524d25f344a6f8e5421d7fec4b31c P 2ece774932e8411aad70a335d13c8e65: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:27:28.429749 10331 mvcc.cc:205] Tried to move back new op lower bound from 7316331103926427648 to 7316331103626973184. Current Snapshot: MvccSnapshot[applied={T|T < 7316331103926427648}]
I20260808 18:27:28.430578 10330 mvcc.cc:205] Tried to move back new op lower bound from 7316331103926427648 to 7316331103626973184. Current Snapshot: MvccSnapshot[applied={T|T < 7316331103926427648}]
I20260808 18:27:28.436184 10249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.432140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54524 (local address 127.7.164.65:36909)
0808 18:27:28.432262 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:28.432267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:28.432462 (+ 195us) server_negotiation.cc:407] Connection header received
0808 18:27:28.432598 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:28.432601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:28.432651 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:28.432736 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:28.432984 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.433574 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.434768 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.435029 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.435117 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.435400 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:28.435805 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:28.435891 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:28.435991 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 18:27:28.436040 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:27:28.446733 10254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.443370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:45465 (local address 127.7.164.66:44659)
0808 18:27:28.443491 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:28.443496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:28.443624 (+ 128us) server_negotiation.cc:407] Connection header received
0808 18:27:28.443779 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:28.443782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:28.443832 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:28.443927 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:28.444175 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.444665 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.445626 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.445854 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.445972 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.446301 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:28.446366 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:28.446451 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:28.446544 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 18:27:28.446593 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:27:28.446766 10348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.443246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:44659 (local address 127.7.164.65:45465)
0808 18:27:28.443536 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:28.443554 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:28.443672 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:28.443936 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:28.443939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:28.443947 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:27:28.444068 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:28.444074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.444782 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.444786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:28.445496 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:28.445506 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.445978 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.445981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:28.446082 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.446123 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:27:28.446460 (+ 337us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:27:28.446469 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 18:27:28.446534 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 18:27:28.446611 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":81,"threads_started":1}
I20260808 18:27:28.453907 9939 raft_consensus.cc:1261] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07 [term 1 FOLLOWER]: Refusing update from remote peer 647495edaa1f493b838a4e9687d6b948: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:28.454310 10155 raft_consensus.cc:1261] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65 [term 1 FOLLOWER]: Refusing update from remote peer 647495edaa1f493b838a4e9687d6b948: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:28.454442 10300 consensus_queue.cc:1035] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51c5740b6ca44d1182f3522a5d79bd07" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:28.454690 10265 consensus_queue.cc:1035] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ece774932e8411aad70a335d13c8e65" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 35587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:28.479803 10249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.475510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:47127 (local address 127.7.164.65:36909)
0808 18:27:28.475640 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:28.475644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:28.476107 (+ 463us) server_negotiation.cc:407] Connection header received
0808 18:27:28.476185 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:28.476188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:28.476244 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:28.476398 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:28.476804 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.477509 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.478682 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.478915 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.479028 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.479357 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:28.479437 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:28.479523 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:28.479611 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 18:27:28.479658 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:27:28.479831 10356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.475493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:36909 (local address 127.7.164.66:47127)
0808 18:27:28.476000 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:28.476013 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:28.476105 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:28.476409 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:28.476412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:28.476420 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:27:28.476685 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:28.476695 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.477595 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.477601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:28.478556 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:28.478568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.479075 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.479079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:28.479163 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.479207 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:27:28.479528 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:27:28.479537 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 18:27:28.479602 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 18:27:28.479680 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":212,"threads_started":1}
I20260808 18:27:28.506656 9476 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:28.506839 9476 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:28.509318 9476 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cda07c0b897d45f18b513897b9e8ddd9: Sending DeleteTablet for 3 replicas of tablet ac83c0df93f0462eb78bfda16fcc9c93
I20260808 18:27:28.510321 9907 tablet_service.cc:1514] Processing DeleteTablet for tablet ac83c0df93f0462eb78bfda16fcc9c93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:28 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:57382
I20260808 18:27:28.510329 10125 tablet_service.cc:1514] Processing DeleteTablet for tablet ac83c0df93f0462eb78bfda16fcc9c93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:28 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:41334
I20260808 18:27:28.510169 9697 tablet_service.cc:1514] Processing DeleteTablet for tablet ac83c0df93f0462eb78bfda16fcc9c93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:28 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:54518
I20260808 18:27:28.514432 10364 tablet_replica.cc:330] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65: stopping tablet replica
I20260808 18:27:28.514616 10364 raft_consensus.cc:2229] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:28.514786 10364 raft_consensus.cc:2258] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:28.515545 10364 ts_tablet_manager.cc:1905] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:28.517460 10367 tablet_replica.cc:330] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07: stopping tablet replica
I20260808 18:27:28.517599 10367 raft_consensus.cc:2229] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:28.517712 10367 raft_consensus.cc:2258] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:28.517939 10368 tablet_replica.cc:330] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948: stopping tablet replica
I20260808 18:27:28.518066 10368 raft_consensus.cc:2229] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:27:28.519029 10367 ts_tablet_manager.cc:1905] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:28.519986 10364 ts_tablet_manager.cc:1918] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:27:28.520542 10364 log.cc:1196] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/ac83c0df93f0462eb78bfda16fcc9c93
I20260808 18:27:28.520915 10364 ts_tablet_manager.cc:1939] T ac83c0df93f0462eb78bfda16fcc9c93 P 2ece774932e8411aad70a335d13c8e65: Deleting consensus metadata
I20260808 18:27:28.521757 10367 ts_tablet_manager.cc:1918] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:27:28.521842 10367 log.cc:1196] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/ac83c0df93f0462eb78bfda16fcc9c93
I20260808 18:27:28.522118 10367 ts_tablet_manager.cc:1939] T ac83c0df93f0462eb78bfda16fcc9c93 P 51c5740b6ca44d1182f3522a5d79bd07: Deleting consensus metadata
I20260808 18:27:28.522796 10368 raft_consensus.cc:2258] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:28.522943 9458 catalog_manager.cc:4841] TS 51c5740b6ca44d1182f3522a5d79bd07 (127.7.164.66:44659): tablet ac83c0df93f0462eb78bfda16fcc9c93 (table test-table [id=2634e39d70c94cbd93c832071b7999a0]) successfully deleted
I20260808 18:27:28.523046 9452 catalog_manager.cc:4841] TS 2ece774932e8411aad70a335d13c8e65 (127.7.164.67:35587): tablet ac83c0df93f0462eb78bfda16fcc9c93 (table test-table [id=2634e39d70c94cbd93c832071b7999a0]) successfully deleted
I20260808 18:27:28.523561 10368 ts_tablet_manager.cc:1905] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:28.526077 10368 ts_tablet_manager.cc:1918] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:27:28.526180 10368 log.cc:1196] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213640155572-7825-0/minicluster-data/ts-0/wal/wals/ac83c0df93f0462eb78bfda16fcc9c93
I20260808 18:27:28.526510 10368 ts_tablet_manager.cc:1939] T ac83c0df93f0462eb78bfda16fcc9c93 P 647495edaa1f493b838a4e9687d6b948: Deleting consensus metadata
I20260808 18:27:28.527737 9453 catalog_manager.cc:4841] TS 647495edaa1f493b838a4e9687d6b948 (127.7.164.65:36909): tablet ac83c0df93f0462eb78bfda16fcc9c93 (table test-table [id=2634e39d70c94cbd93c832071b7999a0]) successfully deleted
Aug 08 18:27:28 dist-test-slave-jztv krb5kdc[9415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213647, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.164.65@KRBTEST.COM
I20260808 18:27:28.537077 10249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.511935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54526 (local address 127.7.164.65:36909)
0808 18:27:28.512043 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:28.512047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:28.520465 (+ 8418us) server_negotiation.cc:407] Connection header received
0808 18:27:28.521265 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:28.521268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:28.521321 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:28.521372 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:28.523990 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.524745 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.526399 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.526766 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.526881 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.531558 (+ 4677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:28.531581 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:28.531631 (+ 50us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:28.531695 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:28.534802 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:28.535409 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:28.535424 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:28.535426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:28.535573 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:28.535896 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:28.535899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:28.535901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:28.536214 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:28.536478 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:28.536618 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 18:27:28.536746 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 18:27:28.537698 9780 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54526
I20260808 18:27:28.549649 10375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:28.539546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41686 (local address 127.7.164.126:37137)
0808 18:27:28.540106 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:28.540110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:28.540426 (+ 316us) server_negotiation.cc:407] Connection header received
0808 18:27:28.540478 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:28.540480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:28.540533 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:28.540610 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:28.541563 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.542438 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.543607 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:28.543984 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:28.544076 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:28.545920 (+ 1844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:28.545924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:28.545927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:28.545956 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:28.547951 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:28.548363 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:28.548368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:28.548369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:28.548458 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:28.548737 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:28.548741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:28.548742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:28.548936 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:28.549083 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:28.549332 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 18:27:28.549477 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":66,"threads_started":1}
W20260808 18:27:28.550067 9476 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41686
I20260808 18:27:28.551565 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 9587
I20260808 18:27:28.566919 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 9800
I20260808 18:27:28.582640 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 10023
I20260808 18:27:28.595903 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 9432
2026-08-08T18:27:28Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3150 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:28 dist-test-slave-jztv krb5kdc[10382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:28 dist-test-slave-jztv krb5kdc[10382](info): set up 2 sockets
Aug 08 18:27:28 dist-test-slave-jztv krb5kdc[10382](info): commencing operation
krb5kdc: starting...
W20260808 18:27:29.827340 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:27:29 dist-test-slave-jztv krb5kdc[10382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:27:29.957392 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:37929
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:42697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:37929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:30.088523 10403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:30.088822 10403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:30.088918 10403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:30.095826 10403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:30.095923 10403 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 18:27:30.095952 10403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:30.095983 10403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:30.096016 10403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:30.096050 10403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:30.102955 10403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:42697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:37929
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:37929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:30.104166 10403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:30.105248 10403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:30.114149 10412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:30.114168 10414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:30.114151 10411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:30.114383 10403 server_base.cc:1034] running on GCE node
I20260808 18:27:30.115188 10403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:30.116313 10403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:30.117519 10403 hybrid_clock.cc:648] HybridClock initialized: now 1786213650117509 us; error 34 us; skew 500 ppm
Aug 08 18:27:30 dist-test-slave-jztv krb5kdc[10382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213650, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:30.120568 10403 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:27:30.121508 10403 webserver.cc:460] Webserver started at http://127.7.164.126:45397/ using document root <none> and password file <none>
I20260808 18:27:30.122104 10403 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:30.122205 10403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:30.122458 10403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:30.124692 10403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "847df9f57c114776adf82064b301361a"
format_stamp: "Formatted at 2026-08-08 18:27:30 on dist-test-slave-jztv"
server_key: "5c8f2a1d13eb424702b22d816bd2a4f1"
server_key_iv: "a1ee8609534ce5c39824d1717d8232dc"
server_key_version: "encryptionkey@0"
I20260808 18:27:30.125301 10403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "847df9f57c114776adf82064b301361a"
format_stamp: "Formatted at 2026-08-08 18:27:30 on dist-test-slave-jztv"
server_key: "5c8f2a1d13eb424702b22d816bd2a4f1"
server_key_iv: "a1ee8609534ce5c39824d1717d8232dc"
server_key_version: "encryptionkey@0"
I20260808 18:27:30.129262 10403 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 18:27:30.131807 10426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:30.133090 10403 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:27:30.133237 10403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "847df9f57c114776adf82064b301361a"
format_stamp: "Formatted at 2026-08-08 18:27:30 on dist-test-slave-jztv"
server_key: "5c8f2a1d13eb424702b22d816bd2a4f1"
server_key_iv: "a1ee8609534ce5c39824d1717d8232dc"
server_key_version: "encryptionkey@0"
I20260808 18:27:30.133359 10403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:30.161253 10403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:30.164757 10403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:30.165000 10403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:30.173024 10403 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:37929
I20260808 18:27:30.173126 10515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:37929 every 8 connection(s)
I20260808 18:27:30.174069 10403 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:30.175246 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 10403
I20260808 18:27:30.175338 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:30.175629 7825 external_mini_cluster.cc:1428] Setting key 76a5003739c1686d289807ab41f88edb
I20260808 18:27:30.177381 10516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:30.183071 10516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a: Bootstrap starting.
Aug 08 18:27:30 dist-test-slave-jztv krb5kdc[10382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:30.185798 10516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:30.187120 10516 log.cc:824] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:30.189452 10516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a: No bootstrap required, opened a new log
I20260808 18:27:30.190117 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.177087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55418 (local address 127.7.164.126:37929)
0808 18:27:30.177597 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:30.177616 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:30.177654 (+ 38us) server_negotiation.cc:407] Connection header received
0808 18:27:30.178378 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:30.178397 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:30.178736 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:30.179083 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:30.179910 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.180894 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.181765 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.182297 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.182477 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:30.185265 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:30.185270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:30.185282 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:30.185312 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:30.187115 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.187636 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.187643 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.187651 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.187802 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.188124 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.188126 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.188127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.188562 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:30.188698 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:30.189046 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 18:27:30.189347 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":143,"threads_started":1}
I20260808 18:27:30.192885 10516 raft_consensus.cc:348] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847df9f57c114776adf82064b301361a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37929 } }
I20260808 18:27:30.193138 10516 raft_consensus.cc:374] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:30.193219 10516 raft_consensus.cc:729] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847df9f57c114776adf82064b301361a, State: Initialized, Role: FOLLOWER
I20260808 18:27:30.193639 10516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847df9f57c114776adf82064b301361a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37929 } }
I20260808 18:27:30.193794 10516 raft_consensus.cc:388] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:30.193882 10516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:30.194010 10516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:30.195114 10516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847df9f57c114776adf82064b301361a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37929 } }
I20260808 18:27:30.195492 10516 leader_election.cc:302] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 847df9f57c114776adf82064b301361a; no voters:
I20260808 18:27:30.195818 10516 leader_election.cc:288] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:30.195986 10523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:30.196228 10523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [term 1 LEADER]: Becoming Leader. State: Replica: 847df9f57c114776adf82064b301361a, State: Running, Role: LEADER
I20260808 18:27:30.196539 10523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847df9f57c114776adf82064b301361a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37929 } }
I20260808 18:27:30.197064 10516 sys_catalog.cc:564] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:30.198297 10525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 847df9f57c114776adf82064b301361a. Latest consensus state: current_term: 1 leader_uuid: "847df9f57c114776adf82064b301361a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847df9f57c114776adf82064b301361a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37929 } } }
I20260808 18:27:30.198448 10525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:30.198734 10524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "847df9f57c114776adf82064b301361a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847df9f57c114776adf82064b301361a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37929 } } }
I20260808 18:27:30.198856 10524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:30.198822 10538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:30.202178 10538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:30.208176 10538 catalog_manager.cc:1347] Generated new cluster ID: de6ea456cb384d9a96376df199230ca7
I20260808 18:27:30.208249 10538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:30.229331 10538 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:30.230360 10538 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:30.239008 10538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a: Generated new TSK 0
I20260808 18:27:30.239741 10538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:27:30.274343 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:37929
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:42697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:30.392706 10556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:30.393008 10556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:30.393090 10556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:30.397149 10556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:30.397243 10556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:30.397315 10556 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 18:27:30.397360 10556 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:27:30.397428 10556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:27:30.402870 10556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:42697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.164.126:37929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:30.404091 10556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:30.405153 10556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:30.414778 10565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:30.414778 10569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:30.414789 10567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:30.415180 10556 server_base.cc:1034] running on GCE node
I20260808 18:27:30.415616 10556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:30.416216 10556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:30.417426 10556 hybrid_clock.cc:648] HybridClock initialized: now 1786213650417398 us; error 44 us; skew 500 ppm
Aug 08 18:27:30 dist-test-slave-jztv krb5kdc[10382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213650, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:30.424029 10556 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:27:30.424911 10556 webserver.cc:460] Webserver started at http://127.7.164.65:42303/ using document root <none> and password file <none>
I20260808 18:27:30.425439 10556 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:30.425500 10556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:30.425671 10556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:30.427743 10556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "43017a05aa454ed7bfef6c18f744770b"
format_stamp: "Formatted at 2026-08-08 18:27:30 on dist-test-slave-jztv"
server_key: "acf04fe4c1a9eecfc4c9e4014b3b2586"
server_key_iv: "629991dfa801ad2c81c22d4f243ebd6a"
server_key_version: "encryptionkey@0"
I20260808 18:27:30.428272 10556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "43017a05aa454ed7bfef6c18f744770b"
format_stamp: "Formatted at 2026-08-08 18:27:30 on dist-test-slave-jztv"
server_key: "acf04fe4c1a9eecfc4c9e4014b3b2586"
server_key_iv: "629991dfa801ad2c81c22d4f243ebd6a"
server_key_version: "encryptionkey@0"
I20260808 18:27:30.432090 10556 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:27:30.434587 10577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:30.435802 10556 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:27:30.435902 10556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "43017a05aa454ed7bfef6c18f744770b"
format_stamp: "Formatted at 2026-08-08 18:27:30 on dist-test-slave-jztv"
server_key: "acf04fe4c1a9eecfc4c9e4014b3b2586"
server_key_iv: "629991dfa801ad2c81c22d4f243ebd6a"
server_key_version: "encryptionkey@0"
I20260808 18:27:30.435989 10556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:30.446321 10556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:30.449568 10556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:30.449772 10556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:30.451345 10556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:30.451395 10556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:30.451437 10556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:30.451490 10556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:27:30 dist-test-slave-jztv krb5kdc[10382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213650, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:30.463045 10556 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:41905
I20260808 18:27:30.463364 10744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:41905 every 8 connection(s)
I20260808 18:27:30.464215 10556 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:27:30.466044 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.452207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:41865 (local address 127.7.164.126:37929)
0808 18:27:30.452337 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:30.452342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:30.453459 (+ 1117us) server_negotiation.cc:407] Connection header received
0808 18:27:30.454801 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:30.454806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:30.454873 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:30.454975 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:30.456459 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.457024 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.458330 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.458680 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.458802 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:30.462061 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:30.462066 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:30.462072 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:30.462106 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:30.463856 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.464469 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.464471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.464473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.464555 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.464981 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.464985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.464990 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.465229 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:30.465309 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:30.465797 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 18:27:30.465916 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:27:30.466877 10591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.452753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37929 (local address 127.7.164.65:41865)
0808 18:27:30.453275 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:30.453318 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:30.454536 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:30.455215 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:30.455227 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:30.455644 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:30.456301 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:30.456318 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.457162 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.457166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:30.458158 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:30.458168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.458901 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.458909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:30.459134 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:30.459866 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:30.459892 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:30.461831 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:30.463985 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:30.463992 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:30.464007 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:30.464367 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:30.464659 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:30.464666 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:30.464669 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:30.464848 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:30.465324 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:30.465329 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:30.465581 (+ 252us) client_negotiation.cc:769] Sending connection context
0808 18:27:30.465741 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 18:27:30.465996 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":144,"threads_started":1}
I20260808 18:27:30.468259 10745 heartbeater.cc:347] Connected to a master server at 127.7.164.126:37929
I20260808 18:27:30.468446 10745 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:30.469045 10745 heartbeater.cc:511] Master 127.7.164.126:37929 requested a full tablet report, sending...
I20260808 18:27:30.470839 10450 ts_manager.cc:194] Registered new tserver with Master: 43017a05aa454ed7bfef6c18f744770b (127.7.164.65:41905)
I20260808 18:27:30.472254 10450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:41865
I20260808 18:27:30.472368 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 10556
I20260808 18:27:30.472486 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:27:30.472805 7825 external_mini_cluster.cc:1428] Setting key 86da65ceeb83c4e5eee3ce2b61110fac
I20260808 18:27:30.478574 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.470796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:58351 (local address 127.7.164.126:37929)
0808 18:27:30.470906 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:30.470909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:30.471153 (+ 244us) server_negotiation.cc:407] Connection header received
0808 18:27:30.471263 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:30.471265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:30.471305 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:30.471370 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:30.471867 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.472564 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.473670 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.473993 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.474115 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:30.475310 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:30.475314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:30.475317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:30.475341 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:30.476832 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.477304 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.477307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.477309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.477398 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.477686 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.477691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.477693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.477883 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:30.477995 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:30.478298 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 18:27:30.478416 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 18:27:30.478592 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.470682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37929 (local address 127.7.164.65:58351)
0808 18:27:30.471053 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:30.471069 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:30.471180 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:30.471415 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:30.471418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:30.471632 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:30.471754 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:30.471761 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.472693 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.472697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:30.473523 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:30.473557 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.474121 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.474125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:30.474251 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:30.474819 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:30.474842 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:30.475165 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:30.476979 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:30.476983 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:30.476985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:30.477194 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:30.477494 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:30.477498 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:30.477500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:30.477590 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:30.477986 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:30.477990 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:30.478091 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 18:27:30.478205 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 18:27:30.478381 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"spinlock_wait_cycles":63360,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:27:30.505147 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:37929
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:42697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:30.621125 10755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:30.621392 10755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:30.621438 10755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:30.625483 10755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:30.625550 10755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:30.625590 10755 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 18:27:30.625604 10755 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:27:30.625670 10755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:27:30.632067 10755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:42697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.164.126:37929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:30.633198 10755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:30.634383 10755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:30.644754 10765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:30.644767 10767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:30.644765 10763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:30.645051 10755 server_base.cc:1034] running on GCE node
I20260808 18:27:30.645628 10755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:30.646394 10755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:30.647588 10755 hybrid_clock.cc:648] HybridClock initialized: now 1786213650647545 us; error 57 us; skew 500 ppm
Aug 08 18:27:30 dist-test-slave-jztv krb5kdc[10382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213650, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:30.650645 10755 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:27:30.651654 10755 webserver.cc:460] Webserver started at http://127.7.164.66:33163/ using document root <none> and password file <none>
I20260808 18:27:30.652233 10755 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:30.652297 10755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:30.652488 10755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:30.654739 10755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "9854c914faa74db8ba78b1400df4a424"
format_stamp: "Formatted at 2026-08-08 18:27:30 on dist-test-slave-jztv"
server_key: "7827e17fdbc43da07bd39639acd41dd2"
server_key_iv: "9ab4bdd5b1aca8067c2c5dadfee8cd0a"
server_key_version: "encryptionkey@0"
I20260808 18:27:30.655313 10755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "9854c914faa74db8ba78b1400df4a424"
format_stamp: "Formatted at 2026-08-08 18:27:30 on dist-test-slave-jztv"
server_key: "7827e17fdbc43da07bd39639acd41dd2"
server_key_iv: "9ab4bdd5b1aca8067c2c5dadfee8cd0a"
server_key_version: "encryptionkey@0"
I20260808 18:27:30.659348 10755 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:27:30.662083 10774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:30.663414 10755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:27:30.663574 10755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "9854c914faa74db8ba78b1400df4a424"
format_stamp: "Formatted at 2026-08-08 18:27:30 on dist-test-slave-jztv"
server_key: "7827e17fdbc43da07bd39639acd41dd2"
server_key_iv: "9ab4bdd5b1aca8067c2c5dadfee8cd0a"
server_key_version: "encryptionkey@0"
I20260808 18:27:30.663699 10755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:30.684926 10755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:30.688000 10755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:30.688211 10755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:30.689810 10755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:30.689888 10755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:30.689963 10755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:30.689988 10755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:27:30 dist-test-slave-jztv krb5kdc[10382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213650, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:30.702170 10755 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:40525
I20260808 18:27:30.702312 10956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:40525 every 8 connection(s)
I20260808 18:27:30.703208 10755 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:27:30.704953 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.690565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:57691 (local address 127.7.164.126:37929)
0808 18:27:30.690700 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:30.690704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:30.692431 (+ 1727us) server_negotiation.cc:407] Connection header received
0808 18:27:30.693666 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:30.693669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:30.693730 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:30.693824 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:30.695508 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.696124 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.697480 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.697765 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.697868 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:30.700840 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:30.700843 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:30.700845 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:30.700873 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:30.702539 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.703150 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.703152 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.703154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.703235 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.703658 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.703662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.703664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.703949 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:30.704066 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:30.704646 (+ 580us) server_negotiation.cc:299] Negotiation successful
0808 18:27:30.704775 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:27:30.705662 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.691673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37929 (local address 127.7.164.66:57691)
0808 18:27:30.692253 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:30.692299 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:30.693449 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:30.694016 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:30.694028 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:30.694572 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:30.695319 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:30.695343 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.696261 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.696265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:30.697349 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:30.697360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.697932 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.697939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:30.698198 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:30.698820 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:30.698841 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:30.700656 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:30.702670 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:30.702675 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:30.702688 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:30.703036 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:30.703380 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:30.703382 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:30.703384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:30.703517 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:30.704063 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:30.704069 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:30.704380 (+ 311us) client_negotiation.cc:769] Sending connection context
0808 18:27:30.704597 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 18:27:30.704898 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":144,"threads_started":1}
I20260808 18:27:30.706826 10957 heartbeater.cc:347] Connected to a master server at 127.7.164.126:37929
I20260808 18:27:30.706986 10957 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:30.707491 10957 heartbeater.cc:511] Master 127.7.164.126:37929 requested a full tablet report, sending...
I20260808 18:27:30.708607 10451 ts_manager.cc:194] Registered new tserver with Master: 9854c914faa74db8ba78b1400df4a424 (127.7.164.66:40525)
I20260808 18:27:30.709426 10451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:57691
I20260808 18:27:30.713014 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 10755
I20260808 18:27:30.713145 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:27:30.713487 7825 external_mini_cluster.cc:1428] Setting key 520dcb55f1ee178a51f9bc1386fe37f8
I20260808 18:27:30.716311 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.708942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37929 (local address 127.7.164.66:54517)
0808 18:27:30.709254 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:30.709267 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:30.709364 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:30.709733 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:30.709738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:30.709951 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:30.710047 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:30.710053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.711075 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.711079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:30.711661 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:30.711669 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.712093 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.712096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:30.712163 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:30.712672 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:30.712683 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:30.713004 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:30.714767 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:30.714771 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:30.714773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:30.715001 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:30.715321 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:30.715325 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:30.715326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:30.715404 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:30.715925 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:30.715927 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:30.716001 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 18:27:30.716079 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 18:27:30.716180 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":97,"threads_started":1}
I20260808 18:27:30.716527 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.709031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:54517 (local address 127.7.164.126:37929)
0808 18:27:30.709140 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:30.709143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:30.709359 (+ 216us) server_negotiation.cc:407] Connection header received
0808 18:27:30.709453 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:30.709455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:30.709493 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:30.709605 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:30.710177 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.710923 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.711775 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.711981 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.712069 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:30.713146 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:30.713150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:30.713156 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:30.713185 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:30.714682 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.715102 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.715106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.715107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.715209 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.715500 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.715505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.715507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.715816 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:30.715928 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:30.716169 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 18:27:30.716304 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:27:30.743227 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:37929
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:42697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:30.860157 10971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:30.860440 10971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:30.860539 10971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:30.865108 10971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:30.865175 10971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:30.865218 10971 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 18:27:30.865283 10971 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:27:30.865345 10971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:27:30.871556 10971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:42697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.164.126:37929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:30.872666 10971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:30.873755 10971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:30.883867 10979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:30.883877 10980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:30.883910 10982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:30.884227 10971 server_base.cc:1034] running on GCE node
I20260808 18:27:30.884624 10971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:30.885216 10971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:30.886373 10971 hybrid_clock.cc:648] HybridClock initialized: now 1786213650886342 us; error 43 us; skew 500 ppm
Aug 08 18:27:30 dist-test-slave-jztv krb5kdc[10382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213650, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:30.889071 10971 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:27:30.889947 10971 webserver.cc:460] Webserver started at http://127.7.164.67:34183/ using document root <none> and password file <none>
I20260808 18:27:30.890575 10971 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:30.890641 10971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:30.890856 10971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:30.892865 10971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "f6bd03cadfec4a209f12443f5de9af02"
format_stamp: "Formatted at 2026-08-08 18:27:30 on dist-test-slave-jztv"
server_key: "81c1082beab4e9a9dc99ff2e46bca054"
server_key_iv: "8982bb0afa8c42a106b6ef89761a5d72"
server_key_version: "encryptionkey@0"
I20260808 18:27:30.893462 10971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "f6bd03cadfec4a209f12443f5de9af02"
format_stamp: "Formatted at 2026-08-08 18:27:30 on dist-test-slave-jztv"
server_key: "81c1082beab4e9a9dc99ff2e46bca054"
server_key_iv: "8982bb0afa8c42a106b6ef89761a5d72"
server_key_version: "encryptionkey@0"
I20260808 18:27:30.897330 10971 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:27:30.900120 10996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:30.901386 10971 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:30.901527 10971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "f6bd03cadfec4a209f12443f5de9af02"
format_stamp: "Formatted at 2026-08-08 18:27:30 on dist-test-slave-jztv"
server_key: "81c1082beab4e9a9dc99ff2e46bca054"
server_key_iv: "8982bb0afa8c42a106b6ef89761a5d72"
server_key_version: "encryptionkey@0"
I20260808 18:27:30.901644 10971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:30.919261 10971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:30.922899 10971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:30.923120 10971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:30.924688 10971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:30.924765 10971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:30.924839 10971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:30.924881 10971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:27:30 dist-test-slave-jztv krb5kdc[10382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213650, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:30.936010 10971 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:39173
I20260808 18:27:30.936182 11159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:39173 every 8 connection(s)
I20260808 18:27:30.936997 10971 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:27:30.939177 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.925426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:54845 (local address 127.7.164.126:37929)
0808 18:27:30.925613 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:30.925619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:30.927073 (+ 1454us) server_negotiation.cc:407] Connection header received
0808 18:27:30.928290 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:30.928294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:30.928353 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:30.928462 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:30.929785 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.930559 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.931813 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.932051 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.932169 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:30.934918 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:30.934921 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:30.934923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:30.934948 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:30.936687 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.937195 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.937200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.937201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.937292 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.937754 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.937757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.937758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.938012 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:30.938141 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:30.938879 (+ 738us) server_negotiation.cc:299] Negotiation successful
0808 18:27:30.939007 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 18:27:30.940033 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.926359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37929 (local address 127.7.164.67:54845)
0808 18:27:30.926925 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:30.926975 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:30.928084 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:30.928631 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:30.928638 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:30.929157 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:30.929634 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:30.929648 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.930689 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.930692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:30.931692 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:30.931702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.932222 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.932226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:30.932399 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:30.933008 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:30.933024 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:30.934732 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:30.936799 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:30.936804 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:30.936818 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:30.937087 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:30.937408 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:30.937411 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:30.937414 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:30.937619 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:30.938154 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:30.938162 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:30.938568 (+ 406us) client_negotiation.cc:769] Sending connection context
0808 18:27:30.938795 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 18:27:30.939092 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":129,"threads_started":1}
I20260808 18:27:30.940462 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 10971
I20260808 18:27:30.940618 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:27:30.940944 7825 external_mini_cluster.cc:1428] Setting key abeb2201c09ec383f6b3d5046c968a7e
I20260808 18:27:30.942551 11162 heartbeater.cc:347] Connected to a master server at 127.7.164.126:37929
I20260808 18:27:30.942705 11162 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:30.943284 11162 heartbeater.cc:511] Master 127.7.164.126:37929 requested a full tablet report, sending...
I20260808 18:27:30.944378 10450 ts_manager.cc:194] Registered new tserver with Master: f6bd03cadfec4a209f12443f5de9af02 (127.7.164.67:39173)
I20260808 18:27:30.945142 10450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:54845
I20260808 18:27:30.951965 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.944793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37929 (local address 127.7.164.67:43453)
0808 18:27:30.945080 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:30.945096 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:30.945200 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:30.945618 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:30.945622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:30.945862 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:30.945975 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:30.945982 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.947015 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.947018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:30.947582 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:30.947590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.948034 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.948037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:30.948120 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:30.948574 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:30.948585 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:30.948912 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:30.950667 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:30.950670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:30.950671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:30.950868 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:30.951121 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:30.951123 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:30.951124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:30.951202 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:30.951554 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:30.951557 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:30.951634 (+ 77us) client_negotiation.cc:769] Sending connection context
0808 18:27:30.951717 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 18:27:30.951823 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":122,"threads_started":1}
I20260808 18:27:30.951970 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.944855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:43453 (local address 127.7.164.126:37929)
0808 18:27:30.944995 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:30.945000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:30.945152 (+ 152us) server_negotiation.cc:407] Connection header received
0808 18:27:30.945338 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:30.945341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:30.945389 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:30.945498 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:30.946107 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.946864 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.947697 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.947904 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.947985 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:30.949060 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:30.949063 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:30.949065 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:30.949087 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:30.950547 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.950966 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.950969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.950970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.951041 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.951278 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.951280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.951281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.951462 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:30.951536 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:30.951779 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 18:27:30.951867 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:27:30.955117 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 18:27:30 dist-test-slave-jztv krb5kdc[10382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:27:30 dist-test-slave-jztv krb5kdc[10382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:30.982760 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.974345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55426 (local address 127.7.164.126:37929)
0808 18:27:30.974495 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:30.974499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:30.974511 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:27:30.974552 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:30.974555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:30.974593 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:30.974689 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:30.975436 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.975951 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.976788 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:30.977013 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:30.977123 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:30.979584 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:30.979587 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:30.979589 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:30.979614 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:30.981007 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.981454 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.981458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.981460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.981550 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:30.981832 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:30.981836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:30.981838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:30.982027 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:30.982167 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:30.982449 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 18:27:30.982586 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 18:27:30.985925 10450 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:27:30.988945 10450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:27:31.003659 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.997724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:41905 (local address 127.0.0.1:44154)
0808 18:27:30.998137 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:30.998196 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:30.998379 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:30.999375 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:30.999383 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:30.999412 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:30.999564 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:30.999570 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.000745 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.000751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.001969 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.001980 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.003073 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.003077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.003186 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.003230 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 18:27:31.003351 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 18:27:31.003496 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":114,"threads_started":1}
I20260808 18:27:31.003679 11188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.997857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44154 (local address 127.7.164.65:41905)
0808 18:27:30.998840 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:30.998848 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:30.998875 (+ 27us) server_negotiation.cc:407] Connection header received
0808 18:27:30.998959 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:30.998965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:30.999174 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:30.999732 (+ 558us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:30.999862 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.000625 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.002108 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.002944 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.003067 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.003215 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:31.003400 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 18:27:31.003521 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":859,"thread_start_us":98,"threads_started":1}
I20260808 18:27:31.004027 11189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.998061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:40525 (local address 127.0.0.1:58730)
0808 18:27:30.998541 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:30.998554 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:30.998686 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:30.999711 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:30.999714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:30.999733 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:30.999837 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:30.999853 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.001274 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.001277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.002438 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.002447 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.003522 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.003526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.003675 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.003710 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 18:27:31.003777 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 18:27:31.003852 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":46,"thread_start_us":104,"threads_started":1}
I20260808 18:27:31.004047 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.998543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:39173 (local address 127.0.0.1:37076)
0808 18:27:30.998894 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:30.998909 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:30.998989 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:30.999850 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:30.999856 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:30.999877 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:31.000026 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.000033 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.001473 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.001477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.002426 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.002437 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.003537 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.003541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.003702 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.003735 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 18:27:31.003795 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 18:27:31.003869 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":41,"thread_start_us":99,"threads_started":1}
I20260808 18:27:31.004576 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.998976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37076 (local address 127.7.164.67:39173)
0808 18:27:30.999434 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:30.999445 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:30.999465 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:27:30.999531 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:30.999535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:30.999702 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:31.000186 (+ 484us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:31.000319 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.001364 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.002582 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.003384 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.004124 (+ 740us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.004280 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:31.004327 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 18:27:31.004410 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":120,"threads_started":1}
I20260808 18:27:31.005095 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:30.998463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58730 (local address 127.7.164.66:40525)
0808 18:27:30.999235 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:30.999246 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:30.999268 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:27:30.999364 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:30.999370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:30.999564 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:30.999723 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:30.999967 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.001134 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.002575 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.003395 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.004322 (+ 927us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.004770 (+ 448us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:31.004871 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 18:27:31.004963 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":121,"threads_started":1}
I20260808 18:27:31.007207 10649 tablet_service.cc:1467] Processing CreateTablet for tablet 287746917d534369b188e6d922c0b258 (DEFAULT_TABLE table=test-table [id=cca751bff75b482ca2c1978e8ee9ff3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:31.007336 11070 tablet_service.cc:1467] Processing CreateTablet for tablet 287746917d534369b188e6d922c0b258 (DEFAULT_TABLE table=test-table [id=cca751bff75b482ca2c1978e8ee9ff3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:31.007546 10854 tablet_service.cc:1467] Processing CreateTablet for tablet 287746917d534369b188e6d922c0b258 (DEFAULT_TABLE table=test-table [id=cca751bff75b482ca2c1978e8ee9ff3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:31.008324 10649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 287746917d534369b188e6d922c0b258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:31.008391 10854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 287746917d534369b188e6d922c0b258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:31.008404 11070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 287746917d534369b188e6d922c0b258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:31.015136 11193 tablet_bootstrap.cc:492] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b: Bootstrap starting.
I20260808 18:27:31.015910 11194 tablet_bootstrap.cc:492] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02: Bootstrap starting.
I20260808 18:27:31.016750 11197 tablet_bootstrap.cc:492] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424: Bootstrap starting.
I20260808 18:27:31.017537 11193 tablet_bootstrap.cc:654] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:31.018064 11194 tablet_bootstrap.cc:654] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:31.018497 11197 tablet_bootstrap.cc:654] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:31.018493 11193 log.cc:824] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:31.018910 11194 log.cc:824] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:31.019346 11197 log.cc:824] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:31.020612 11193 tablet_bootstrap.cc:492] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b: No bootstrap required, opened a new log
I20260808 18:27:31.020854 11193 ts_tablet_manager.cc:1397] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 18:27:31.020908 11194 tablet_bootstrap.cc:492] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02: No bootstrap required, opened a new log
I20260808 18:27:31.021075 11194 ts_tablet_manager.cc:1397] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:27:31.021279 11197 tablet_bootstrap.cc:492] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424: No bootstrap required, opened a new log
I20260808 18:27:31.021642 11197 ts_tablet_manager.cc:1397] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:27:31.025738 11193 raft_consensus.cc:348] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } }
I20260808 18:27:31.026036 11193 raft_consensus.cc:374] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:31.026131 11193 raft_consensus.cc:729] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43017a05aa454ed7bfef6c18f744770b, State: Initialized, Role: FOLLOWER
I20260808 18:27:31.026342 11197 raft_consensus.cc:348] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } }
I20260808 18:27:31.026439 11194 raft_consensus.cc:348] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } }
I20260808 18:27:31.026618 11197 raft_consensus.cc:374] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:31.026672 11197 raft_consensus.cc:729] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9854c914faa74db8ba78b1400df4a424, State: Initialized, Role: FOLLOWER
I20260808 18:27:31.026640 11193 consensus_queue.cc:260] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } }
I20260808 18:27:31.026724 11194 raft_consensus.cc:374] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:31.026779 11194 raft_consensus.cc:729] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bd03cadfec4a209f12443f5de9af02, State: Initialized, Role: FOLLOWER
I20260808 18:27:31.027134 11197 consensus_queue.cc:260] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } }
I20260808 18:27:31.027230 11194 consensus_queue.cc:260] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } }
I20260808 18:27:31.027688 11193 ts_tablet_manager.cc:1428] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260808 18:27:31.028083 10745 heartbeater.cc:503] Master 127.7.164.126:37929 was elected leader, sending a full tablet report...
I20260808 18:27:31.028175 11197 ts_tablet_manager.cc:1428] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:27:31.028168 11194 ts_tablet_manager.cc:1428] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 18:27:31.028474 10957 heartbeater.cc:503] Master 127.7.164.126:37929 was elected leader, sending a full tablet report...
I20260808 18:27:31.028483 11162 heartbeater.cc:503] Master 127.7.164.126:37929 was elected leader, sending a full tablet report...
I20260808 18:27:31.070511 11204 raft_consensus.cc:482] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:27:31.070758 11204 raft_consensus.cc:504] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } }
I20260808 18:27:31.071933 11204 leader_election.cc:288] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43017a05aa454ed7bfef6c18f744770b (127.7.164.65:41905), f6bd03cadfec4a209f12443f5de9af02 (127.7.164.67:39173)
I20260808 18:27:31.076666 11188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.072404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:59835 (local address 127.7.164.65:41905)
0808 18:27:31.072570 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:31.072574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:31.072593 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:27:31.072749 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:31.072752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:31.072823 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:31.072902 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:31.073284 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.074149 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.075542 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.076163 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.076277 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.076320 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:31.076511 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 18:27:31.076567 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:27:31.076694 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.072297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:41905 (local address 127.7.164.66:59835)
0808 18:27:31.072467 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:31.072488 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:31.072596 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:31.072935 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:31.072938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:31.072965 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:31.073123 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.073132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.074297 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.074301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.075410 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.075421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.076272 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.076275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.076378 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.076406 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 18:27:31.076487 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 18:27:31.076558 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 18:27:31.077348 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "287746917d534369b188e6d922c0b258" candidate_uuid: "9854c914faa74db8ba78b1400df4a424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43017a05aa454ed7bfef6c18f744770b" is_pre_election: true
I20260808 18:27:31.077415 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.072746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:55253 (local address 127.7.164.67:39173)
0808 18:27:31.072886 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:31.072890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:31.073071 (+ 181us) server_negotiation.cc:407] Connection header received
0808 18:27:31.073182 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:31.073187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:31.073249 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:31.073316 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:31.074029 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.074792 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.076203 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.076820 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.076953 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.076995 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:31.077233 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 18:27:31.077304 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:27:31.077433 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.072580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:39173 (local address 127.7.164.66:55253)
0808 18:27:31.072972 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:31.072987 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:31.073084 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:31.073332 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:31.073335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:31.073353 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:31.073907 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.073917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.075038 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.075042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.076068 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.076079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.076967 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.076971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.077077 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.077105 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 18:27:31.077202 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 18:27:31.077270 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":158,"threads_started":1}
I20260808 18:27:31.077728 10681 raft_consensus.cc:2454] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9854c914faa74db8ba78b1400df4a424 in term 0.
I20260808 18:27:31.078303 10783 leader_election.cc:302] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43017a05aa454ed7bfef6c18f744770b, 9854c914faa74db8ba78b1400df4a424; no voters:
I20260808 18:27:31.078413 11099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "287746917d534369b188e6d922c0b258" candidate_uuid: "9854c914faa74db8ba78b1400df4a424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bd03cadfec4a209f12443f5de9af02" is_pre_election: true
I20260808 18:27:31.078653 11204 raft_consensus.cc:2781] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:27:31.078756 11204 raft_consensus.cc:482] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:27:31.078747 11099 raft_consensus.cc:2454] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9854c914faa74db8ba78b1400df4a424 in term 0.
I20260808 18:27:31.078819 11204 raft_consensus.cc:3037] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:31.079937 11204 raft_consensus.cc:504] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } }
I20260808 18:27:31.080289 11204 leader_election.cc:288] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [CANDIDATE]: Term 1 election: Requested vote from peers 43017a05aa454ed7bfef6c18f744770b (127.7.164.65:41905), f6bd03cadfec4a209f12443f5de9af02 (127.7.164.67:39173)
I20260808 18:27:31.080693 11099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "287746917d534369b188e6d922c0b258" candidate_uuid: "9854c914faa74db8ba78b1400df4a424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bd03cadfec4a209f12443f5de9af02"
I20260808 18:27:31.080719 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "287746917d534369b188e6d922c0b258" candidate_uuid: "9854c914faa74db8ba78b1400df4a424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43017a05aa454ed7bfef6c18f744770b"
I20260808 18:27:31.080798 11099 raft_consensus.cc:3037] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:31.080873 10681 raft_consensus.cc:3037] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:31.081923 10681 raft_consensus.cc:2454] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9854c914faa74db8ba78b1400df4a424 in term 1.
I20260808 18:27:31.082165 11099 raft_consensus.cc:2454] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9854c914faa74db8ba78b1400df4a424 in term 1.
I20260808 18:27:31.082278 10783 leader_election.cc:302] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43017a05aa454ed7bfef6c18f744770b, 9854c914faa74db8ba78b1400df4a424; no voters:
I20260808 18:27:31.082587 11204 raft_consensus.cc:2781] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:31.082779 11204 raft_consensus.cc:686] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [term 1 LEADER]: Becoming Leader. State: Replica: 9854c914faa74db8ba78b1400df4a424, State: Running, Role: LEADER
I20260808 18:27:31.083094 11204 consensus_queue.cc:237] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } }
I20260808 18:27:31.086345 10447 catalog_manager.cc:5495] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9854c914faa74db8ba78b1400df4a424 (127.7.164.66). New cstate: current_term: 1 leader_uuid: "9854c914faa74db8ba78b1400df4a424" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } health_report { overall_health: HEALTHY } } }
Aug 08 18:27:31 dist-test-slave-jztv krb5kdc[10382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213650, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:31.136565 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.128190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55434 (local address 127.7.164.126:37929)
0808 18:27:31.128357 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:31.128363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:31.128532 (+ 169us) server_negotiation.cc:407] Connection header received
0808 18:27:31.128701 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:31.128705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:31.128748 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:31.128834 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:31.129527 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.130042 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.130910 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.131130 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.131242 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.133548 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:31.133551 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:31.133553 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:31.133577 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:31.134929 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:31.135372 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:31.135374 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:31.135376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:31.135450 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:31.135692 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:31.135694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:31.135695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:31.135870 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:31.135944 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:31.136239 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 18:27:31.136392 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":48}
I20260808 18:27:31.136598 11220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.128075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37929 (local address 127.0.0.1:55434)
0808 18:27:31.128435 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:31.128447 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:31.128601 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:31.128854 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:31.128857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:31.129294 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:31.129390 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.129398 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.130156 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.130161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.130783 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.130792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.131247 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.131250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.131331 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.131892 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:31.131905 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:31.133402 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:31.135037 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:31.135045 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:31.135051 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:31.135282 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:31.135532 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:31.135534 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:31.135535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:31.135615 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:31.135965 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:31.135971 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:31.136085 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 18:27:31.136180 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 18:27:31.136332 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":128,"threads_started":1}
I20260808 18:27:31.140846 10447 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:55434:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 18:27:31.141606 10447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:27:31.146766 10649 tablet_service.cc:1467] Processing CreateTablet for tablet df35d1d92ba4427989bf5fd2be68fcc8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=56b511b0e76b437d9ee57f139aa9aec6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:27:31.146766 10854 tablet_service.cc:1467] Processing CreateTablet for tablet df35d1d92ba4427989bf5fd2be68fcc8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=56b511b0e76b437d9ee57f139aa9aec6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:27:31.147059 11070 tablet_service.cc:1467] Processing CreateTablet for tablet df35d1d92ba4427989bf5fd2be68fcc8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=56b511b0e76b437d9ee57f139aa9aec6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:27:31.147516 10649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df35d1d92ba4427989bf5fd2be68fcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:31.147509 10854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df35d1d92ba4427989bf5fd2be68fcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:31.147619 11070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df35d1d92ba4427989bf5fd2be68fcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:31.150035 11193 tablet_bootstrap.cc:492] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b: Bootstrap starting.
I20260808 18:27:31.150516 11197 tablet_bootstrap.cc:492] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424: Bootstrap starting.
I20260808 18:27:31.151223 11193 tablet_bootstrap.cc:654] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:31.151562 11197 tablet_bootstrap.cc:654] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:31.152292 11194 tablet_bootstrap.cc:492] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02: Bootstrap starting.
I20260808 18:27:31.152664 11193 tablet_bootstrap.cc:492] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b: No bootstrap required, opened a new log
I20260808 18:27:31.152770 11193 ts_tablet_manager.cc:1397] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:27:31.153327 11194 tablet_bootstrap.cc:654] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:31.153319 11193 raft_consensus.cc:348] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.153463 11193 raft_consensus.cc:374] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:31.153512 11193 raft_consensus.cc:729] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43017a05aa454ed7bfef6c18f744770b, State: Initialized, Role: FOLLOWER
I20260808 18:27:31.153663 11193 consensus_queue.cc:260] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.153990 11203 tablet_replica.cc:439] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } }
I20260808 18:27:31.154182 11203 tablet_replica.cc:442] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:27:31.153997 11193 ts_tablet_manager.cc:1428] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:27:31.154975 11194 tablet_bootstrap.cc:492] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02: No bootstrap required, opened a new log
I20260808 18:27:31.155053 11194 ts_tablet_manager.cc:1397] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:27:31.155488 11194 raft_consensus.cc:348] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.155628 11194 raft_consensus.cc:374] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:31.155669 11194 raft_consensus.cc:729] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bd03cadfec4a209f12443f5de9af02, State: Initialized, Role: FOLLOWER
I20260808 18:27:31.155818 11194 consensus_queue.cc:260] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.156097 11194 ts_tablet_manager.cc:1428] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:31.156106 11207 tablet_replica.cc:439] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } }
I20260808 18:27:31.156275 11207 tablet_replica.cc:442] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:27:31.156991 11197 tablet_bootstrap.cc:492] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424: No bootstrap required, opened a new log
I20260808 18:27:31.157109 11197 ts_tablet_manager.cc:1397] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260808 18:27:31.157645 11197 raft_consensus.cc:348] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.157766 11197 raft_consensus.cc:374] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:31.157812 11197 raft_consensus.cc:729] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9854c914faa74db8ba78b1400df4a424, State: Initialized, Role: FOLLOWER
I20260808 18:27:31.157941 11197 consensus_queue.cc:260] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.158280 11197 ts_tablet_manager.cc:1428] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 18:27:31.158275 11204 tablet_replica.cc:439] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } }
I20260808 18:27:31.158424 11204 tablet_replica.cc:442] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 18:27:31.188138 11163 tablet.cc:2369] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:27:31.204816 10958 tablet.cc:2369] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:27:31.215770 10746 tablet.cc:2369] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:27:31.269567 11207 raft_consensus.cc:482] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:27:31.269726 11207 raft_consensus.cc:504] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.271112 11207 leader_election.cc:288] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43017a05aa454ed7bfef6c18f744770b (127.7.164.65:41905), 9854c914faa74db8ba78b1400df4a424 (127.7.164.66:40525)
I20260808 18:27:31.275267 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.271565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:53443 (local address 127.7.164.66:40525)
0808 18:27:31.271720 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:31.271725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:31.271742 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:27:31.271802 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:31.271806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:31.271862 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:31.271961 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:31.272308 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.272991 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.274274 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.274699 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.274838 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.274894 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:31.275080 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 18:27:31.275154 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 18:27:31.275267 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.271367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:40525 (local address 127.7.164.67:53443)
0808 18:27:31.271533 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:31.271552 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:31.271651 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:31.272011 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:31.272014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:31.272037 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:31.272181 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.272192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.273101 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.273106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.274119 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.274129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.274825 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.274828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.274944 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.274971 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:27:31.275061 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 18:27:31.275133 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":28}
I20260808 18:27:31.275456 11188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.271633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:35677 (local address 127.7.164.65:41905)
0808 18:27:31.271817 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:31.271820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:31.271974 (+ 154us) server_negotiation.cc:407] Connection header received
0808 18:27:31.272066 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:31.272069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:31.272115 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:31.272204 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:31.272551 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.273277 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.274472 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.274888 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.275029 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.275078 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:31.275279 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 18:27:31.275348 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260808 18:27:31.275484 11228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.271396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:41905 (local address 127.7.164.67:35677)
0808 18:27:31.271875 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:31.271890 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:31.271984 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:31.272224 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:31.272226 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:31.272262 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:31.272386 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.272393 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.273389 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.273394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.274347 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.274357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.275038 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.275042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.275143 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.275164 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:27:31.275260 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 18:27:31.275329 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":3,"thread_start_us":121,"threads_started":1}
I20260808 18:27:31.275951 10880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df35d1d92ba4427989bf5fd2be68fcc8" candidate_uuid: "f6bd03cadfec4a209f12443f5de9af02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9854c914faa74db8ba78b1400df4a424" is_pre_election: true
I20260808 18:27:31.276041 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df35d1d92ba4427989bf5fd2be68fcc8" candidate_uuid: "f6bd03cadfec4a209f12443f5de9af02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43017a05aa454ed7bfef6c18f744770b" is_pre_election: true
I20260808 18:27:31.276207 10880 raft_consensus.cc:2454] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bd03cadfec4a209f12443f5de9af02 in term 0.
I20260808 18:27:31.276229 10681 raft_consensus.cc:2454] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bd03cadfec4a209f12443f5de9af02 in term 0.
I20260808 18:27:31.276665 11002 leader_election.cc:302] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 43017a05aa454ed7bfef6c18f744770b, 9854c914faa74db8ba78b1400df4a424, f6bd03cadfec4a209f12443f5de9af02; no voters:
I20260808 18:27:31.276973 11207 raft_consensus.cc:2781] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:27:31.277055 11207 raft_consensus.cc:482] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:27:31.277112 11207 raft_consensus.cc:3037] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:31.278117 11207 raft_consensus.cc:504] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.278491 11207 leader_election.cc:288] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [CANDIDATE]: Term 1 election: Requested vote from peers 43017a05aa454ed7bfef6c18f744770b (127.7.164.65:41905), 9854c914faa74db8ba78b1400df4a424 (127.7.164.66:40525)
I20260808 18:27:31.278880 10880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df35d1d92ba4427989bf5fd2be68fcc8" candidate_uuid: "f6bd03cadfec4a209f12443f5de9af02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9854c914faa74db8ba78b1400df4a424"
I20260808 18:27:31.278892 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df35d1d92ba4427989bf5fd2be68fcc8" candidate_uuid: "f6bd03cadfec4a209f12443f5de9af02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43017a05aa454ed7bfef6c18f744770b"
I20260808 18:27:31.279011 10681 raft_consensus.cc:3037] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:31.279011 10880 raft_consensus.cc:3037] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:31.280162 10880 raft_consensus.cc:2454] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bd03cadfec4a209f12443f5de9af02 in term 1.
I20260808 18:27:31.280162 10681 raft_consensus.cc:2454] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bd03cadfec4a209f12443f5de9af02 in term 1.
I20260808 18:27:31.280479 11001 leader_election.cc:302] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9854c914faa74db8ba78b1400df4a424, f6bd03cadfec4a209f12443f5de9af02; no voters:
I20260808 18:27:31.280722 11207 raft_consensus.cc:2781] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:31.280946 11207 raft_consensus.cc:686] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [term 1 LEADER]: Becoming Leader. State: Replica: f6bd03cadfec4a209f12443f5de9af02, State: Running, Role: LEADER
I20260808 18:27:31.281270 11207 consensus_queue.cc:237] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.283493 11229 tablet_replica.cc:439] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02: TxnStatusTablet state changed. Reason: New leader f6bd03cadfec4a209f12443f5de9af02. Latest consensus state: current_term: 1 leader_uuid: "f6bd03cadfec4a209f12443f5de9af02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } }
I20260808 18:27:31.283617 11229 tablet_replica.cc:442] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:27:31.283923 11234 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 18:27:31.284130 11234 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 18:27:31.284731 10447 catalog_manager.cc:5495] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6bd03cadfec4a209f12443f5de9af02 (127.7.164.67). New cstate: current_term: 1 leader_uuid: "f6bd03cadfec4a209f12443f5de9af02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } health_report { overall_health: HEALTHY } } }
I20260808 18:27:31.313302 11220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.309240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:39173 (local address 127.0.0.1:37092)
0808 18:27:31.309392 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:31.309410 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:31.309517 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:31.309859 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:31.309861 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:31.309868 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:27:31.309972 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.309976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.310990 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.310993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.311582 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.311589 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.312170 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.312173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.312244 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.312292 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:27:31.313047 (+ 755us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:27:31.313060 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 18:27:31.313113 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 18:27:31.313184 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 18:27:31.313340 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.309364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37092 (local address 127.7.164.67:39173)
0808 18:27:31.309497 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:31.309501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:31.309518 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:27:31.309665 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:31.309668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:31.309734 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:31.309840 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:31.310080 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.310887 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.311704 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.312056 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.312161 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.312435 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:31.312949 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:31.313074 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:31.313133 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 18:27:31.313191 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:27:31.318943 10681 raft_consensus.cc:1261] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Refusing update from remote peer f6bd03cadfec4a209f12443f5de9af02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:31.318943 10880 raft_consensus.cc:1261] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Refusing update from remote peer f6bd03cadfec4a209f12443f5de9af02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:31.319634 11229 consensus_queue.cc:1035] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:31.319810 11207 consensus_queue.cc:1035] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:31.322918 11204 tablet_replica.cc:439] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424: TxnStatusTablet state changed. Reason: New leader f6bd03cadfec4a209f12443f5de9af02. Latest consensus state: current_term: 1 leader_uuid: "f6bd03cadfec4a209f12443f5de9af02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } }
I20260808 18:27:31.323037 11204 tablet_replica.cc:442] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:27:31.324870 11203 tablet_replica.cc:439] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b: TxnStatusTablet state changed. Reason: New leader f6bd03cadfec4a209f12443f5de9af02. Latest consensus state: current_term: 1 leader_uuid: "f6bd03cadfec4a209f12443f5de9af02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } }
I20260808 18:27:31.324998 11203 tablet_replica.cc:442] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:27:31.326437 11229 tablet_replica.cc:439] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6bd03cadfec4a209f12443f5de9af02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } }
I20260808 18:27:31.326575 11229 tablet_replica.cc:442] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:27:31.326699 11239 mvcc.cc:205] Tried to move back new op lower bound from 7316331115795197952 to 7316331115650568192. Current Snapshot: MvccSnapshot[applied={T|T < 7316331115795197952}]
I20260808 18:27:31.326746 11204 tablet_replica.cc:439] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6bd03cadfec4a209f12443f5de9af02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } }
I20260808 18:27:31.326836 11204 tablet_replica.cc:442] T df35d1d92ba4427989bf5fd2be68fcc8 P 9854c914faa74db8ba78b1400df4a424: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:27:31.328967 11207 tablet_replica.cc:439] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6bd03cadfec4a209f12443f5de9af02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } }
I20260808 18:27:31.329073 11207 tablet_replica.cc:442] T df35d1d92ba4427989bf5fd2be68fcc8 P f6bd03cadfec4a209f12443f5de9af02: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:27:31.329919 11203 tablet_replica.cc:439] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6bd03cadfec4a209f12443f5de9af02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } } }
I20260808 18:27:31.330037 11203 tablet_replica.cc:442] T df35d1d92ba4427989bf5fd2be68fcc8 P 43017a05aa454ed7bfef6c18f744770b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:27:31.331307 11240 mvcc.cc:205] Tried to move back new op lower bound from 7316331115795197952 to 7316331115650568192. Current Snapshot: MvccSnapshot[applied={T|T < 7316331115795197952 or (T in {7316331115795197952})}]
I20260808 18:27:31.331983 11243 mvcc.cc:205] Tried to move back new op lower bound from 7316331115795197952 to 7316331115650568192. Current Snapshot: MvccSnapshot[applied={T|T < 7316331115795197952}]
I20260808 18:27:31.338485 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.334276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58740 (local address 127.7.164.66:40525)
0808 18:27:31.334406 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:31.334409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:31.334422 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:27:31.334580 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:31.334582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:31.334630 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:31.334713 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:31.334977 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.335503 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.336724 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.337002 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.337140 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.337470 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:31.338027 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:31.338133 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:31.338242 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 18:27:31.338305 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:27:31.350303 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.346790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:38587 (local address 127.7.164.67:39173)
0808 18:27:31.346964 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:31.346968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:31.347096 (+ 128us) server_negotiation.cc:407] Connection header received
0808 18:27:31.347263 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:31.347266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:31.347325 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:31.347406 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:31.347701 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.348225 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.349113 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.349348 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.349465 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.349797 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:31.349876 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:31.349966 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:31.350082 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 18:27:31.350133 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:27:31.350337 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.346651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:39173 (local address 127.7.164.66:38587)
0808 18:27:31.346997 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:31.347013 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:31.347150 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:31.347447 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:31.347450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:31.347458 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:27:31.347591 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.347596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.348346 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.348351 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.348985 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.348994 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.349489 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.349493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.349593 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.349635 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:27:31.349971 (+ 336us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:27:31.349981 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 18:27:31.350069 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 18:27:31.350147 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":117,"threads_started":1}
I20260808 18:27:31.358333 10681 raft_consensus.cc:1261] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Refusing update from remote peer 9854c914faa74db8ba78b1400df4a424: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:31.358414 11099 raft_consensus.cc:1261] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Refusing update from remote peer 9854c914faa74db8ba78b1400df4a424: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:31.358856 11204 consensus_queue.cc:1035] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:31.359046 11210 consensus_queue.cc:1035] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:31.382647 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.378851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:40525 (local address 127.7.164.67:60595)
0808 18:27:31.379064 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:31.379081 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:31.379167 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:31.379595 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:31.379599 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:31.379608 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:27:31.379720 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.379726 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.380476 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.380479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.381421 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:31.381431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.381862 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.381866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:31.381956 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.382016 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:27:31.382352 (+ 336us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:27:31.382359 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 18:27:31.382427 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 18:27:31.382507 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260808 18:27:31.383198 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:31.378936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:60595 (local address 127.7.164.66:40525)
0808 18:27:31.379349 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:31.379355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:31.379367 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:27:31.379406 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:31.379409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:31.379474 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:31.379553 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:31.379834 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.380363 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.381542 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:31.381750 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:31.381831 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:31.382142 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:31.382259 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:31.382330 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:31.383021 (+ 691us) server_negotiation.cc:299] Negotiation successful
0808 18:27:31.383078 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311}
I20260808 18:27:31.409732 10451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:31.409929 10451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:31.412843 10451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a: Sending DeleteTablet for 3 replicas of tablet 287746917d534369b188e6d922c0b258
I20260808 18:27:31.413661 11070 tablet_service.cc:1514] Processing DeleteTablet for tablet 287746917d534369b188e6d922c0b258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:31 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:37076
I20260808 18:27:31.413743 10854 tablet_service.cc:1514] Processing DeleteTablet for tablet 287746917d534369b188e6d922c0b258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:31 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:58730
I20260808 18:27:31.413815 10649 tablet_service.cc:1514] Processing DeleteTablet for tablet 287746917d534369b188e6d922c0b258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:31 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:44154
I20260808 18:27:31.414433 11275 tablet_replica.cc:330] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02: stopping tablet replica
I20260808 18:27:31.414790 11275 raft_consensus.cc:2229] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:31.415033 11275 raft_consensus.cc:2258] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:31.414942 10451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:27:31.415364 10451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:27:31.417918 11275 ts_tablet_manager.cc:1905] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:31.420805 10649 tablet_service.cc:1467] Processing CreateTablet for tablet 013899fab697425289e4f50ed8a7301f (DEFAULT_TABLE table=test-table [id=ff92a6bac82848b0ac8cda988de8c877]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:31.421159 10649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 013899fab697425289e4f50ed8a7301f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:31.421288 11275 ts_tablet_manager.cc:1918] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:27:31.421453 11275 log.cc:1196] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/287746917d534369b188e6d922c0b258
I20260808 18:27:31.421844 11275 ts_tablet_manager.cc:1939] T 287746917d534369b188e6d922c0b258 P f6bd03cadfec4a209f12443f5de9af02: Deleting consensus metadata
I20260808 18:27:31.423094 10432 catalog_manager.cc:4841] TS f6bd03cadfec4a209f12443f5de9af02 (127.7.164.67:39173): tablet 287746917d534369b188e6d922c0b258 (table test-table [id=cca751bff75b482ca2c1978e8ee9ff3a]) successfully deleted
I20260808 18:27:31.423789 11193 tablet_bootstrap.cc:492] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b: Bootstrap starting.
I20260808 18:27:31.424046 11070 tablet_service.cc:1467] Processing CreateTablet for tablet 013899fab697425289e4f50ed8a7301f (DEFAULT_TABLE table=test-table [id=ff92a6bac82848b0ac8cda988de8c877]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:31.424295 11070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 013899fab697425289e4f50ed8a7301f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:31.424729 11193 tablet_bootstrap.cc:654] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:31.426003 11193 tablet_bootstrap.cc:492] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b: No bootstrap required, opened a new log
I20260808 18:27:31.426083 11193 ts_tablet_manager.cc:1397] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:31.426378 11278 tablet_replica.cc:330] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b: stopping tablet replica
I20260808 18:27:31.426558 11193 raft_consensus.cc:348] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.426656 11193 raft_consensus.cc:374] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:31.426689 11193 raft_consensus.cc:729] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43017a05aa454ed7bfef6c18f744770b, State: Initialized, Role: FOLLOWER
I20260808 18:27:31.426849 11193 consensus_queue.cc:260] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.427155 11193 ts_tablet_manager.cc:1428] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:31.427188 10854 tablet_service.cc:1467] Processing CreateTablet for tablet 013899fab697425289e4f50ed8a7301f (DEFAULT_TABLE table=test-table [id=ff92a6bac82848b0ac8cda988de8c877]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:31.426680 11278 raft_consensus.cc:2229] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:31.427500 10854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 013899fab697425289e4f50ed8a7301f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:31.427551 11278 raft_consensus.cc:2258] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:31.427928 11277 tablet_replica.cc:330] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424: stopping tablet replica
I20260808 18:27:31.428224 11277 raft_consensus.cc:2229] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:27:31.428619 11277 raft_consensus.cc:2258] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:31.431176 11194 tablet_bootstrap.cc:492] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02: Bootstrap starting.
I20260808 18:27:31.432229 11194 tablet_bootstrap.cc:654] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:31.432511 11277 ts_tablet_manager.cc:1905] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:31.433243 11278 ts_tablet_manager.cc:1905] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:31.435515 11277 ts_tablet_manager.cc:1918] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:27:31.435668 11277 log.cc:1196] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/287746917d534369b188e6d922c0b258
I20260808 18:27:31.436046 11277 ts_tablet_manager.cc:1939] T 287746917d534369b188e6d922c0b258 P 9854c914faa74db8ba78b1400df4a424: Deleting consensus metadata
I20260808 18:27:31.437362 10430 catalog_manager.cc:4841] TS 9854c914faa74db8ba78b1400df4a424 (127.7.164.66:40525): tablet 287746917d534369b188e6d922c0b258 (table test-table [id=cca751bff75b482ca2c1978e8ee9ff3a]) successfully deleted
I20260808 18:27:31.437844 11197 tablet_bootstrap.cc:492] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424: Bootstrap starting.
I20260808 18:27:31.438907 11197 tablet_bootstrap.cc:654] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:31.439366 11278 ts_tablet_manager.cc:1918] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:27:31.439527 11278 log.cc:1196] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/wal/wals/287746917d534369b188e6d922c0b258
I20260808 18:27:31.439914 11278 ts_tablet_manager.cc:1939] T 287746917d534369b188e6d922c0b258 P 43017a05aa454ed7bfef6c18f744770b: Deleting consensus metadata
I20260808 18:27:31.441232 10432 catalog_manager.cc:4841] TS 43017a05aa454ed7bfef6c18f744770b (127.7.164.65:41905): tablet 287746917d534369b188e6d922c0b258 (table test-table [id=cca751bff75b482ca2c1978e8ee9ff3a]) successfully deleted
I20260808 18:27:31.443115 11197 tablet_bootstrap.cc:492] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424: No bootstrap required, opened a new log
I20260808 18:27:31.443104 11194 tablet_bootstrap.cc:492] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02: No bootstrap required, opened a new log
I20260808 18:27:31.443204 11194 ts_tablet_manager.cc:1397] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260808 18:27:31.443218 11197 ts_tablet_manager.cc:1397] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 18:27:31.443694 11194 raft_consensus.cc:348] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.443694 11197 raft_consensus.cc:348] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.443827 11197 raft_consensus.cc:374] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:31.443825 11194 raft_consensus.cc:374] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:31.443868 11197 raft_consensus.cc:729] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9854c914faa74db8ba78b1400df4a424, State: Initialized, Role: FOLLOWER
I20260808 18:27:31.443868 11194 raft_consensus.cc:729] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bd03cadfec4a209f12443f5de9af02, State: Initialized, Role: FOLLOWER
I20260808 18:27:31.444013 11197 consensus_queue.cc:260] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.444003 11194 consensus_queue.cc:260] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.444296 11194 ts_tablet_manager.cc:1428] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:27:31.444304 11197 ts_tablet_manager.cc:1428] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:31.580556 11270 raft_consensus.cc:482] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:27:31.580710 11270 raft_consensus.cc:504] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.581213 11270 leader_election.cc:288] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43017a05aa454ed7bfef6c18f744770b (127.7.164.65:41905), f6bd03cadfec4a209f12443f5de9af02 (127.7.164.67:39173)
I20260808 18:27:31.581820 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013899fab697425289e4f50ed8a7301f" candidate_uuid: "9854c914faa74db8ba78b1400df4a424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43017a05aa454ed7bfef6c18f744770b" is_pre_election: true
I20260808 18:27:31.581843 11099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013899fab697425289e4f50ed8a7301f" candidate_uuid: "9854c914faa74db8ba78b1400df4a424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bd03cadfec4a209f12443f5de9af02" is_pre_election: true
I20260808 18:27:31.581993 10679 raft_consensus.cc:2454] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9854c914faa74db8ba78b1400df4a424 in term 0.
I20260808 18:27:31.582010 11099 raft_consensus.cc:2454] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9854c914faa74db8ba78b1400df4a424 in term 0.
I20260808 18:27:31.582324 10783 leader_election.cc:302] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9854c914faa74db8ba78b1400df4a424, f6bd03cadfec4a209f12443f5de9af02; no voters:
I20260808 18:27:31.582576 11270 raft_consensus.cc:2781] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:27:31.582652 11270 raft_consensus.cc:482] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:27:31.582681 11270 raft_consensus.cc:3037] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:31.583747 11270 raft_consensus.cc:504] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.584102 11270 leader_election.cc:288] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [CANDIDATE]: Term 1 election: Requested vote from peers 43017a05aa454ed7bfef6c18f744770b (127.7.164.65:41905), f6bd03cadfec4a209f12443f5de9af02 (127.7.164.67:39173)
I20260808 18:27:31.584471 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013899fab697425289e4f50ed8a7301f" candidate_uuid: "9854c914faa74db8ba78b1400df4a424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43017a05aa454ed7bfef6c18f744770b"
I20260808 18:27:31.584613 10679 raft_consensus.cc:3037] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:31.584578 11099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013899fab697425289e4f50ed8a7301f" candidate_uuid: "9854c914faa74db8ba78b1400df4a424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bd03cadfec4a209f12443f5de9af02"
I20260808 18:27:31.584734 11099 raft_consensus.cc:3037] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:31.585647 10679 raft_consensus.cc:2454] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9854c914faa74db8ba78b1400df4a424 in term 1.
I20260808 18:27:31.585680 11099 raft_consensus.cc:2454] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9854c914faa74db8ba78b1400df4a424 in term 1.
I20260808 18:27:31.585964 10783 leader_election.cc:302] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43017a05aa454ed7bfef6c18f744770b, 9854c914faa74db8ba78b1400df4a424; no voters:
I20260808 18:27:31.586202 11270 raft_consensus.cc:2781] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:31.586370 11270 raft_consensus.cc:686] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [term 1 LEADER]: Becoming Leader. State: Replica: 9854c914faa74db8ba78b1400df4a424, State: Running, Role: LEADER
I20260808 18:27:31.586535 11270 consensus_queue.cc:237] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:31.588367 10447 catalog_manager.cc:5495] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9854c914faa74db8ba78b1400df4a424 (127.7.164.66). New cstate: current_term: 1 leader_uuid: "9854c914faa74db8ba78b1400df4a424" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } health_report { overall_health: UNKNOWN } } }
I20260808 18:27:31.641773 10681 raft_consensus.cc:1261] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Refusing update from remote peer 9854c914faa74db8ba78b1400df4a424: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:31.642071 11099 raft_consensus.cc:1261] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Refusing update from remote peer 9854c914faa74db8ba78b1400df4a424: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:31.642544 11270 consensus_queue.cc:1035] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:31.642740 11204 consensus_queue.cc:1035] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 18:27:31.656617 11240 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 18:27:33.221038 10447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:33.221197 10447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:33.223832 10447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a: Sending DeleteTablet for 3 replicas of tablet 013899fab697425289e4f50ed8a7301f
I20260808 18:27:33.224496 10649 tablet_service.cc:1514] Processing DeleteTablet for tablet 013899fab697425289e4f50ed8a7301f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:33 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:44154
I20260808 18:27:33.224511 11070 tablet_service.cc:1514] Processing DeleteTablet for tablet 013899fab697425289e4f50ed8a7301f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:33 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:37076
I20260808 18:27:33.224541 10854 tablet_service.cc:1514] Processing DeleteTablet for tablet 013899fab697425289e4f50ed8a7301f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:33 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:58730
I20260808 18:27:33.224908 11324 tablet_replica.cc:330] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424: stopping tablet replica
I20260808 18:27:33.225082 11324 raft_consensus.cc:2229] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:27:33.225329 11324 raft_consensus.cc:2258] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:33.225627 11322 tablet_replica.cc:330] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b: stopping tablet replica
I20260808 18:27:33.225793 11322 raft_consensus.cc:2229] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:33.225926 11322 raft_consensus.cc:2258] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:33.226620 11324 ts_tablet_manager.cc:1905] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:33.227449 11323 tablet_replica.cc:330] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02: stopping tablet replica
I20260808 18:27:33.227584 11323 raft_consensus.cc:2229] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:33.227703 11323 raft_consensus.cc:2258] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:33.228258 11322 ts_tablet_manager.cc:1905] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:33.228873 11323 ts_tablet_manager.cc:1905] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:33.229658 11324 ts_tablet_manager.cc:1918] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:27:33.229746 11324 log.cc:1196] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/013899fab697425289e4f50ed8a7301f
I20260808 18:27:33.230062 11324 ts_tablet_manager.cc:1939] T 013899fab697425289e4f50ed8a7301f P 9854c914faa74db8ba78b1400df4a424: Deleting consensus metadata
I20260808 18:27:33.230981 10430 catalog_manager.cc:4841] TS 9854c914faa74db8ba78b1400df4a424 (127.7.164.66:40525): tablet 013899fab697425289e4f50ed8a7301f (table test-table [id=ff92a6bac82848b0ac8cda988de8c877]) successfully deleted
I20260808 18:27:33.231765 11322 ts_tablet_manager.cc:1918] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:27:33.231854 11322 log.cc:1196] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/wal/wals/013899fab697425289e4f50ed8a7301f
I20260808 18:27:33.231819 11323 ts_tablet_manager.cc:1918] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:27:33.231931 11323 log.cc:1196] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/013899fab697425289e4f50ed8a7301f
I20260808 18:27:33.232157 11322 ts_tablet_manager.cc:1939] T 013899fab697425289e4f50ed8a7301f P 43017a05aa454ed7bfef6c18f744770b: Deleting consensus metadata
I20260808 18:27:33.232184 11323 ts_tablet_manager.cc:1939] T 013899fab697425289e4f50ed8a7301f P f6bd03cadfec4a209f12443f5de9af02: Deleting consensus metadata
I20260808 18:27:33.232911 10432 catalog_manager.cc:4841] TS f6bd03cadfec4a209f12443f5de9af02 (127.7.164.67:39173): tablet 013899fab697425289e4f50ed8a7301f (table test-table [id=ff92a6bac82848b0ac8cda988de8c877]) successfully deleted
I20260808 18:27:33.233120 10432 catalog_manager.cc:4841] TS 43017a05aa454ed7bfef6c18f744770b (127.7.164.65:41905): tablet 013899fab697425289e4f50ed8a7301f (table test-table [id=ff92a6bac82848b0ac8cda988de8c877]) successfully deleted
Aug 08 18:27:33 dist-test-slave-jztv krb5kdc[10382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213653, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:27:33 dist-test-slave-jztv krb5kdc[10382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213653, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:33.257002 11339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:33.247750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55436 (local address 127.7.164.126:37929)
0808 18:27:33.248040 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:33.248050 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:33.248144 (+ 94us) server_negotiation.cc:407] Connection header received
0808 18:27:33.248327 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:33.248331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:33.248389 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:33.248508 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:33.249224 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.249796 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.250737 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.250979 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.251093 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:33.253744 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:33.253747 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:33.253749 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:33.253776 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:33.255263 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:33.255695 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:33.255700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:33.255704 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:33.255799 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:33.256084 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:33.256088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:33.256090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:33.256280 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:33.256444 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:33.256680 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 18:27:33.256812 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":98,"threads_started":1}
I20260808 18:27:33.259671 10447 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:27:33.260089 10447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:27:33.265110 10649 tablet_service.cc:1467] Processing CreateTablet for tablet b5a0ac62cf5b4f2897f07db24a54661b (DEFAULT_TABLE table=test-table [id=4d1bdea426974697a118b39acf5afaa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:33.265228 10854 tablet_service.cc:1467] Processing CreateTablet for tablet b5a0ac62cf5b4f2897f07db24a54661b (DEFAULT_TABLE table=test-table [id=4d1bdea426974697a118b39acf5afaa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:33.265303 11070 tablet_service.cc:1467] Processing CreateTablet for tablet b5a0ac62cf5b4f2897f07db24a54661b (DEFAULT_TABLE table=test-table [id=4d1bdea426974697a118b39acf5afaa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:33.265467 10649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5a0ac62cf5b4f2897f07db24a54661b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:33.265566 11070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5a0ac62cf5b4f2897f07db24a54661b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:33.265585 10854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5a0ac62cf5b4f2897f07db24a54661b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:33.267782 11340 tablet_bootstrap.cc:492] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424: Bootstrap starting.
I20260808 18:27:33.268494 11342 tablet_bootstrap.cc:492] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02: Bootstrap starting.
I20260808 18:27:33.268894 11340 tablet_bootstrap.cc:654] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:33.268959 11341 tablet_bootstrap.cc:492] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b: Bootstrap starting.
I20260808 18:27:33.269480 11342 tablet_bootstrap.cc:654] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:33.269923 11341 tablet_bootstrap.cc:654] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:33.270201 11340 tablet_bootstrap.cc:492] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424: No bootstrap required, opened a new log
I20260808 18:27:33.270334 11340 ts_tablet_manager.cc:1397] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:27:33.270780 11342 tablet_bootstrap.cc:492] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02: No bootstrap required, opened a new log
I20260808 18:27:33.270879 11342 ts_tablet_manager.cc:1397] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:33.270836 11340 raft_consensus.cc:348] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.270967 11340 raft_consensus.cc:374] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:33.270993 11340 raft_consensus.cc:729] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9854c914faa74db8ba78b1400df4a424, State: Initialized, Role: FOLLOWER
I20260808 18:27:33.271126 11340 consensus_queue.cc:260] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.271440 11340 ts_tablet_manager.cc:1428] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:33.271381 11342 raft_consensus.cc:348] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.271499 11342 raft_consensus.cc:374] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:33.271533 11342 raft_consensus.cc:729] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bd03cadfec4a209f12443f5de9af02, State: Initialized, Role: FOLLOWER
I20260808 18:27:33.271656 11342 consensus_queue.cc:260] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.271730 11341 tablet_bootstrap.cc:492] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b: No bootstrap required, opened a new log
I20260808 18:27:33.271809 11341 ts_tablet_manager.cc:1397] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:27:33.271950 11342 ts_tablet_manager.cc:1428] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:33.272284 11341 raft_consensus.cc:348] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.272418 11341 raft_consensus.cc:374] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:33.272459 11341 raft_consensus.cc:729] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43017a05aa454ed7bfef6c18f744770b, State: Initialized, Role: FOLLOWER
I20260808 18:27:33.272605 11341 consensus_queue.cc:260] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.273046 11341 ts_tablet_manager.cc:1428] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 18:27:33.306088 11348 raft_consensus.cc:482] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:27:33.306299 11348 raft_consensus.cc:504] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.307857 11348 leader_election.cc:288] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9854c914faa74db8ba78b1400df4a424 (127.7.164.66:40525), f6bd03cadfec4a209f12443f5de9af02 (127.7.164.67:39173)
I20260808 18:27:33.312412 11350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:33.308142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:39173 (local address 127.7.164.65:50027)
0808 18:27:33.308607 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:33.308622 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:33.308721 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:33.309025 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:33.309028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:33.309051 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:33.309222 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:33.309233 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.310033 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.310036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:33.311105 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:33.311116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.312018 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.312022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:33.312121 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:33.312142 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:27:33.312208 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 18:27:33.312255 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":50,"thread_start_us":123,"threads_started":1}
I20260808 18:27:33.312724 11351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:33.308163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:40687 (local address 127.7.164.66:40525)
0808 18:27:33.308474 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:33.308485 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:33.308601 (+ 116us) server_negotiation.cc:407] Connection header received
0808 18:27:33.308725 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:33.308728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:33.308789 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:33.308878 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:33.309289 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.309875 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.311116 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.311729 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.311873 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:33.311953 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:33.312483 (+ 530us) server_negotiation.cc:299] Negotiation successful
0808 18:27:33.312592 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":117,"threads_started":1}
I20260808 18:27:33.312424 11349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:33.308045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:40525 (local address 127.7.164.65:40687)
0808 18:27:33.308473 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:33.308489 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:33.308627 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:33.308906 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:33.308909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:33.308933 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:33.309085 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:33.309099 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.310000 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.310004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:33.310979 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:33.310988 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.311874 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.311878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:33.312026 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:33.312054 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 18:27:33.312130 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 18:27:33.312218 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":44,"thread_start_us":138,"threads_started":1}
I20260808 18:27:33.312916 11352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:33.308262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:50027 (local address 127.7.164.67:39173)
0808 18:27:33.308681 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:33.308693 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:33.308708 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:27:33.308818 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:33.308820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:33.308898 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:33.308985 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:33.309358 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.309869 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.311247 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.311773 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.311902 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:33.311961 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:33.312264 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 18:27:33.312426 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":110,"threads_started":1}
I20260808 18:27:33.313328 10880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a0ac62cf5b4f2897f07db24a54661b" candidate_uuid: "43017a05aa454ed7bfef6c18f744770b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9854c914faa74db8ba78b1400df4a424" is_pre_election: true
I20260808 18:27:33.313335 11099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a0ac62cf5b4f2897f07db24a54661b" candidate_uuid: "43017a05aa454ed7bfef6c18f744770b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bd03cadfec4a209f12443f5de9af02" is_pre_election: true
I20260808 18:27:33.313467 11099 raft_consensus.cc:2454] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43017a05aa454ed7bfef6c18f744770b in term 0.
I20260808 18:27:33.313503 10880 raft_consensus.cc:2454] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43017a05aa454ed7bfef6c18f744770b in term 0.
I20260808 18:27:33.313973 10582 leader_election.cc:302] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43017a05aa454ed7bfef6c18f744770b, 9854c914faa74db8ba78b1400df4a424; no voters:
I20260808 18:27:33.314324 11348 raft_consensus.cc:2781] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:27:33.314407 11348 raft_consensus.cc:482] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:27:33.314448 11348 raft_consensus.cc:3037] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:33.315481 11348 raft_consensus.cc:504] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.315822 11348 leader_election.cc:288] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [CANDIDATE]: Term 1 election: Requested vote from peers 9854c914faa74db8ba78b1400df4a424 (127.7.164.66:40525), f6bd03cadfec4a209f12443f5de9af02 (127.7.164.67:39173)
I20260808 18:27:33.316174 10880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a0ac62cf5b4f2897f07db24a54661b" candidate_uuid: "43017a05aa454ed7bfef6c18f744770b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9854c914faa74db8ba78b1400df4a424"
I20260808 18:27:33.316226 11099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a0ac62cf5b4f2897f07db24a54661b" candidate_uuid: "43017a05aa454ed7bfef6c18f744770b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bd03cadfec4a209f12443f5de9af02"
I20260808 18:27:33.316314 10880 raft_consensus.cc:3037] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:33.316336 11099 raft_consensus.cc:3037] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:33.317482 10880 raft_consensus.cc:2454] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43017a05aa454ed7bfef6c18f744770b in term 1.
I20260808 18:27:33.317474 11099 raft_consensus.cc:2454] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43017a05aa454ed7bfef6c18f744770b in term 1.
I20260808 18:27:33.317844 10583 leader_election.cc:302] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43017a05aa454ed7bfef6c18f744770b, f6bd03cadfec4a209f12443f5de9af02; no voters:
I20260808 18:27:33.318037 11348 raft_consensus.cc:2781] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:33.318346 11348 raft_consensus.cc:686] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [term 1 LEADER]: Becoming Leader. State: Replica: 43017a05aa454ed7bfef6c18f744770b, State: Running, Role: LEADER
I20260808 18:27:33.318602 11348 consensus_queue.cc:237] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.320919 10450 catalog_manager.cc:5495] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b reported cstate change: term changed from 0 to 1, leader changed from <none> to 43017a05aa454ed7bfef6c18f744770b (127.7.164.65). New cstate: current_term: 1 leader_uuid: "43017a05aa454ed7bfef6c18f744770b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } health_report { overall_health: UNKNOWN } } }
I20260808 18:27:33.344321 11360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:33.340305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44164 (local address 127.7.164.65:41905)
0808 18:27:33.340673 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:33.340681 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:33.340694 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:27:33.340772 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:33.340774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:33.340835 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:33.340943 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:33.341171 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.341735 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.342743 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.343052 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.343140 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:33.343478 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:33.343951 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:33.344046 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:33.344126 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 18:27:33.344176 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":214,"threads_started":1}
I20260808 18:27:33.354509 11352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:33.351418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:43907 (local address 127.7.164.67:39173)
0808 18:27:33.351547 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:33.351551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:33.351696 (+ 145us) server_negotiation.cc:407] Connection header received
0808 18:27:33.351804 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:33.351806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:33.351854 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:33.351946 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:33.352167 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.352660 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.353523 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.353744 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.353827 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:33.354116 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:33.354187 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:33.354268 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:33.354346 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 18:27:33.354397 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:27:33.354553 11364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:33.351297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:39173 (local address 127.7.164.65:43907)
0808 18:27:33.351616 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:33.351630 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:33.351718 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:33.351953 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:33.351955 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:33.351962 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:27:33.352072 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:33.352077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.352755 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.352757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:33.353393 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:33.353400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.353851 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.353853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:33.353929 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:33.353975 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:27:33.354289 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:27:33.354293 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:27:33.354340 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 18:27:33.354407 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":114,"threads_started":1}
I20260808 18:27:33.360605 11099 raft_consensus.cc:1261] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Refusing update from remote peer 43017a05aa454ed7bfef6c18f744770b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:33.360725 10880 raft_consensus.cc:1261] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Refusing update from remote peer 43017a05aa454ed7bfef6c18f744770b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:33.361186 11348 consensus_queue.cc:1035] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:33.361411 11355 consensus_queue.cc:1035] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:33.380056 11360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:33.376589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:54265 (local address 127.7.164.65:41905)
0808 18:27:33.376709 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:33.376714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:33.376943 (+ 229us) server_negotiation.cc:407] Connection header received
0808 18:27:33.377073 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:33.377076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:33.377134 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:33.377204 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:33.377476 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.378103 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.378968 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.379183 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.379314 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:33.379575 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:33.379660 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:33.379738 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:33.379848 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 18:27:33.379895 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 18:27:33.380115 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:33.376470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:41905 (local address 127.7.164.67:54265)
0808 18:27:33.376817 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:33.376851 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:33.376967 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:33.377223 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:33.377225 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:33.377233 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:27:33.377358 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:33.377365 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.378250 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.378254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:33.378861 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:33.378870 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.379304 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.379307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:33.379412 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:33.379447 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:27:33.379756 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:27:33.379763 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 18:27:33.379842 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 18:27:33.379935 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":123,"threads_started":1}
I20260808 18:27:33.403085 10451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:33.403239 10451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:33.405596 10451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a: Sending DeleteTablet for 3 replicas of tablet b5a0ac62cf5b4f2897f07db24a54661b
I20260808 18:27:33.406270 10854 tablet_service.cc:1514] Processing DeleteTablet for tablet b5a0ac62cf5b4f2897f07db24a54661b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:33 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:58730
I20260808 18:27:33.406410 10649 tablet_service.cc:1514] Processing DeleteTablet for tablet b5a0ac62cf5b4f2897f07db24a54661b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:33 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:44154
I20260808 18:27:33.406486 11324 tablet_replica.cc:330] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424: stopping tablet replica
I20260808 18:27:33.406598 11322 tablet_replica.cc:330] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b: stopping tablet replica
I20260808 18:27:33.406625 11324 raft_consensus.cc:2229] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:33.406724 11322 raft_consensus.cc:2229] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:27:33.406773 11324 raft_consensus.cc:2258] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:33.407070 11322 raft_consensus.cc:2258] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:33.407307 11070 tablet_service.cc:1514] Processing DeleteTablet for tablet b5a0ac62cf5b4f2897f07db24a54661b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:33 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:37076
I20260808 18:27:33.407684 11324 ts_tablet_manager.cc:1905] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:33.408030 11322 ts_tablet_manager.cc:1905] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:33.408417 10451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:27:33.408870 10451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:27:33.409456 11323 tablet_replica.cc:330] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02: stopping tablet replica
I20260808 18:27:33.409602 11323 raft_consensus.cc:2229] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:33.409725 11323 raft_consensus.cc:2258] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:33.410637 11322 ts_tablet_manager.cc:1918] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:27:33.410710 11322 log.cc:1196] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-0/wal/wals/b5a0ac62cf5b4f2897f07db24a54661b
I20260808 18:27:33.410889 11323 ts_tablet_manager.cc:1905] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:33.410986 11322 ts_tablet_manager.cc:1939] T b5a0ac62cf5b4f2897f07db24a54661b P 43017a05aa454ed7bfef6c18f744770b: Deleting consensus metadata
I20260808 18:27:33.411751 10432 catalog_manager.cc:4841] TS 43017a05aa454ed7bfef6c18f744770b (127.7.164.65:41905): tablet b5a0ac62cf5b4f2897f07db24a54661b (table test-table [id=4d1bdea426974697a118b39acf5afaa0]) successfully deleted
I20260808 18:27:33.413457 11323 ts_tablet_manager.cc:1918] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:27:33.413542 11323 log.cc:1196] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/b5a0ac62cf5b4f2897f07db24a54661b
I20260808 18:27:33.413836 11323 ts_tablet_manager.cc:1939] T b5a0ac62cf5b4f2897f07db24a54661b P f6bd03cadfec4a209f12443f5de9af02: Deleting consensus metadata
I20260808 18:27:33.415099 11324 ts_tablet_manager.cc:1918] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:27:33.415227 11324 log.cc:1196] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/b5a0ac62cf5b4f2897f07db24a54661b
I20260808 18:27:33.415236 10854 tablet_service.cc:1467] Processing CreateTablet for tablet 75dfcf0a4fc74eb1a2b890d542749a62 (DEFAULT_TABLE table=test-table [id=4718454891864dc4be5baee8abe3e89c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:33.415742 10854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75dfcf0a4fc74eb1a2b890d542749a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:33.416123 10432 catalog_manager.cc:4841] TS f6bd03cadfec4a209f12443f5de9af02 (127.7.164.67:39173): tablet b5a0ac62cf5b4f2897f07db24a54661b (table test-table [id=4d1bdea426974697a118b39acf5afaa0]) successfully deleted
I20260808 18:27:33.416653 11070 tablet_service.cc:1467] Processing CreateTablet for tablet 75dfcf0a4fc74eb1a2b890d542749a62 (DEFAULT_TABLE table=test-table [id=4718454891864dc4be5baee8abe3e89c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:33.416826 10649 tablet_service.cc:1467] Processing CreateTablet for tablet 75dfcf0a4fc74eb1a2b890d542749a62 (DEFAULT_TABLE table=test-table [id=4718454891864dc4be5baee8abe3e89c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:33.416909 11070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75dfcf0a4fc74eb1a2b890d542749a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:33.417094 10649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75dfcf0a4fc74eb1a2b890d542749a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:33.415740 11324 ts_tablet_manager.cc:1939] T b5a0ac62cf5b4f2897f07db24a54661b P 9854c914faa74db8ba78b1400df4a424: Deleting consensus metadata
I20260808 18:27:33.419204 10430 catalog_manager.cc:4841] TS 9854c914faa74db8ba78b1400df4a424 (127.7.164.66:40525): tablet b5a0ac62cf5b4f2897f07db24a54661b (table test-table [id=4d1bdea426974697a118b39acf5afaa0]) successfully deleted
I20260808 18:27:33.419317 11342 tablet_bootstrap.cc:492] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02: Bootstrap starting.
I20260808 18:27:33.419435 11340 tablet_bootstrap.cc:492] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424: Bootstrap starting.
I20260808 18:27:33.420060 11342 tablet_bootstrap.cc:654] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:33.420260 11340 tablet_bootstrap.cc:654] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:33.421165 11342 tablet_bootstrap.cc:492] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02: No bootstrap required, opened a new log
I20260808 18:27:33.421252 11342 ts_tablet_manager.cc:1397] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:33.421604 11342 raft_consensus.cc:348] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.421677 11340 tablet_bootstrap.cc:492] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424: No bootstrap required, opened a new log
I20260808 18:27:33.421690 11342 raft_consensus.cc:374] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:33.421712 11342 raft_consensus.cc:729] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bd03cadfec4a209f12443f5de9af02, State: Initialized, Role: FOLLOWER
I20260808 18:27:33.421753 11340 ts_tablet_manager.cc:1397] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 18:27:33.421850 11342 consensus_queue.cc:260] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.422125 11342 ts_tablet_manager.cc:1428] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:27:33.422133 11340 raft_consensus.cc:348] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.422260 11340 raft_consensus.cc:374] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:33.422294 11340 raft_consensus.cc:729] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9854c914faa74db8ba78b1400df4a424, State: Initialized, Role: FOLLOWER
I20260808 18:27:33.422415 11340 consensus_queue.cc:260] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.422667 11340 ts_tablet_manager.cc:1428] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:33.423396 11341 tablet_bootstrap.cc:492] T 75dfcf0a4fc74eb1a2b890d542749a62 P 43017a05aa454ed7bfef6c18f744770b: Bootstrap starting.
I20260808 18:27:33.424817 11341 tablet_bootstrap.cc:654] T 75dfcf0a4fc74eb1a2b890d542749a62 P 43017a05aa454ed7bfef6c18f744770b: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:33.426666 11341 tablet_bootstrap.cc:492] T 75dfcf0a4fc74eb1a2b890d542749a62 P 43017a05aa454ed7bfef6c18f744770b: No bootstrap required, opened a new log
I20260808 18:27:33.426787 11341 ts_tablet_manager.cc:1397] T 75dfcf0a4fc74eb1a2b890d542749a62 P 43017a05aa454ed7bfef6c18f744770b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260808 18:27:33.427260 11341 raft_consensus.cc:348] T 75dfcf0a4fc74eb1a2b890d542749a62 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.427388 11341 raft_consensus.cc:374] T 75dfcf0a4fc74eb1a2b890d542749a62 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:33.427420 11341 raft_consensus.cc:729] T 75dfcf0a4fc74eb1a2b890d542749a62 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43017a05aa454ed7bfef6c18f744770b, State: Initialized, Role: FOLLOWER
I20260808 18:27:33.427538 11341 consensus_queue.cc:260] T 75dfcf0a4fc74eb1a2b890d542749a62 P 43017a05aa454ed7bfef6c18f744770b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.427811 11341 ts_tablet_manager.cc:1428] T 75dfcf0a4fc74eb1a2b890d542749a62 P 43017a05aa454ed7bfef6c18f744770b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:27:33.558813 11291 raft_consensus.cc:482] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:27:33.559013 11291 raft_consensus.cc:504] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.559562 11291 leader_election.cc:288] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43017a05aa454ed7bfef6c18f744770b (127.7.164.65:41905), 9854c914faa74db8ba78b1400df4a424 (127.7.164.66:40525)
I20260808 18:27:33.560043 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75dfcf0a4fc74eb1a2b890d542749a62" candidate_uuid: "f6bd03cadfec4a209f12443f5de9af02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43017a05aa454ed7bfef6c18f744770b" is_pre_election: true
I20260808 18:27:33.560035 10880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75dfcf0a4fc74eb1a2b890d542749a62" candidate_uuid: "f6bd03cadfec4a209f12443f5de9af02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9854c914faa74db8ba78b1400df4a424" is_pre_election: true
I20260808 18:27:33.560216 10880 raft_consensus.cc:2454] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bd03cadfec4a209f12443f5de9af02 in term 0.
I20260808 18:27:33.560223 10681 raft_consensus.cc:2454] T 75dfcf0a4fc74eb1a2b890d542749a62 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bd03cadfec4a209f12443f5de9af02 in term 0.
I20260808 18:27:33.560529 11001 leader_election.cc:302] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9854c914faa74db8ba78b1400df4a424, f6bd03cadfec4a209f12443f5de9af02; no voters:
I20260808 18:27:33.560736 11291 raft_consensus.cc:2781] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:27:33.560804 11291 raft_consensus.cc:482] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:27:33.560860 11291 raft_consensus.cc:3037] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:33.561906 11291 raft_consensus.cc:504] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.562359 11291 leader_election.cc:288] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [CANDIDATE]: Term 1 election: Requested vote from peers 43017a05aa454ed7bfef6c18f744770b (127.7.164.65:41905), 9854c914faa74db8ba78b1400df4a424 (127.7.164.66:40525)
I20260808 18:27:33.562635 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75dfcf0a4fc74eb1a2b890d542749a62" candidate_uuid: "f6bd03cadfec4a209f12443f5de9af02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43017a05aa454ed7bfef6c18f744770b"
I20260808 18:27:33.562763 10681 raft_consensus.cc:3037] T 75dfcf0a4fc74eb1a2b890d542749a62 P 43017a05aa454ed7bfef6c18f744770b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:33.562716 10880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75dfcf0a4fc74eb1a2b890d542749a62" candidate_uuid: "f6bd03cadfec4a209f12443f5de9af02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9854c914faa74db8ba78b1400df4a424"
I20260808 18:27:33.562882 10880 raft_consensus.cc:3037] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:33.563836 10880 raft_consensus.cc:2454] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bd03cadfec4a209f12443f5de9af02 in term 1.
I20260808 18:27:33.563984 10681 raft_consensus.cc:2454] T 75dfcf0a4fc74eb1a2b890d542749a62 P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bd03cadfec4a209f12443f5de9af02 in term 1.
I20260808 18:27:33.564208 11001 leader_election.cc:302] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9854c914faa74db8ba78b1400df4a424, f6bd03cadfec4a209f12443f5de9af02; no voters:
I20260808 18:27:33.564459 11291 raft_consensus.cc:2781] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:33.564569 11291 raft_consensus.cc:686] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [term 1 LEADER]: Becoming Leader. State: Replica: f6bd03cadfec4a209f12443f5de9af02, State: Running, Role: LEADER
I20260808 18:27:33.564720 11291 consensus_queue.cc:237] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } }
I20260808 18:27:33.566773 10451 catalog_manager.cc:5495] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6bd03cadfec4a209f12443f5de9af02 (127.7.164.67). New cstate: current_term: 1 leader_uuid: "f6bd03cadfec4a209f12443f5de9af02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6bd03cadfec4a209f12443f5de9af02" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39173 } health_report { overall_health: HEALTHY } } }
I20260808 18:27:33.589363 11352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:33.586583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37104 (local address 127.7.164.67:39173)
0808 18:27:33.586702 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:33.586706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:33.586745 (+ 39us) server_negotiation.cc:407] Connection header received
0808 18:27:33.586855 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:33.586857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:33.586899 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:33.586970 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:33.587189 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.587697 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.588493 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.588697 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.588772 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:33.588990 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:33.589056 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:33.589109 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:33.589215 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 18:27:33.589256 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:27:33.598202 11352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:33.595209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:44437 (local address 127.7.164.67:39173)
0808 18:27:33.595317 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:33.595323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:33.595337 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:27:33.595383 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:33.595386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:33.595419 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:33.595495 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:33.595743 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.596235 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.597095 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.597378 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.597518 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:33.597756 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:33.597830 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:33.597910 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:33.598007 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 18:27:33.598047 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":44}
I20260808 18:27:33.598202 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:33.595008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:39173 (local address 127.7.164.67:44437)
0808 18:27:33.595170 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:33.595189 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:33.595284 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:33.595513 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:33.595515 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:33.595522 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:27:33.595644 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:33.595650 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.596350 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.596353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:33.596965 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:33.596974 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:33.597519 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:33.597522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:33.597606 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:33.597631 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:27:33.597914 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:27:33.597921 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 18:27:33.597994 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 18:27:33.598085 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":19}
I20260808 18:27:33.603873 10880 raft_consensus.cc:1261] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Refusing update from remote peer f6bd03cadfec4a209f12443f5de9af02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:33.603895 10681 raft_consensus.cc:1261] T 75dfcf0a4fc74eb1a2b890d542749a62 P 43017a05aa454ed7bfef6c18f744770b [term 1 FOLLOWER]: Refusing update from remote peer f6bd03cadfec4a209f12443f5de9af02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:33.604403 11298 consensus_queue.cc:1035] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9854c914faa74db8ba78b1400df4a424" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:33.604588 11207 consensus_queue.cc:1035] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43017a05aa454ed7bfef6c18f744770b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 18:27:33.623620 11289 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 18:27:35.192153 10451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:35.192356 10451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:35.195117 10451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 847df9f57c114776adf82064b301361a: Sending DeleteTablet for 3 replicas of tablet 75dfcf0a4fc74eb1a2b890d542749a62
I20260808 18:27:35.195775 10854 tablet_service.cc:1514] Processing DeleteTablet for tablet 75dfcf0a4fc74eb1a2b890d542749a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:35 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:58730
I20260808 18:27:35.195806 10649 tablet_service.cc:1514] Processing DeleteTablet for tablet 75dfcf0a4fc74eb1a2b890d542749a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:35 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:44154
I20260808 18:27:35.195801 11070 tablet_service.cc:1514] Processing DeleteTablet for tablet 75dfcf0a4fc74eb1a2b890d542749a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:35 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:37076
I20260808 18:27:35.196168 11433 tablet_replica.cc:330] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02: stopping tablet replica
I20260808 18:27:35.196192 11427 tablet_replica.cc:330] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424: stopping tablet replica
I20260808 18:27:35.196233 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 10556
I20260808 18:27:35.196322 11427 raft_consensus.cc:2229] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:35.202469 11427 raft_consensus.cc:2258] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:35.203435 11427 ts_tablet_manager.cc:1905] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:35.206476 11427 ts_tablet_manager.cc:1918] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:27:35.206566 11427 log.cc:1196] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/75dfcf0a4fc74eb1a2b890d542749a62
I20260808 18:27:35.206729 11433 raft_consensus.cc:2229] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:27:35.206866 11427 ts_tablet_manager.cc:1939] T 75dfcf0a4fc74eb1a2b890d542749a62 P 9854c914faa74db8ba78b1400df4a424: Deleting consensus metadata
I20260808 18:27:35.207141 11433 raft_consensus.cc:2258] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:35.208297 11433 ts_tablet_manager.cc:1905] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:35.212288 11433 ts_tablet_manager.cc:1918] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:27:35.212411 11433 log.cc:1196] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/75dfcf0a4fc74eb1a2b890d542749a62
I20260808 18:27:35.212656 10430 catalog_manager.cc:4841] TS 9854c914faa74db8ba78b1400df4a424 (127.7.164.66:40525): tablet 75dfcf0a4fc74eb1a2b890d542749a62 (table test-table [id=4718454891864dc4be5baee8abe3e89c]) successfully deleted
I20260808 18:27:35.212749 11433 ts_tablet_manager.cc:1939] T 75dfcf0a4fc74eb1a2b890d542749a62 P f6bd03cadfec4a209f12443f5de9af02: Deleting consensus metadata
I20260808 18:27:35.213491 10432 catalog_manager.cc:4841] TS f6bd03cadfec4a209f12443f5de9af02 (127.7.164.67:39173): tablet 75dfcf0a4fc74eb1a2b890d542749a62 (table test-table [id=4718454891864dc4be5baee8abe3e89c]) successfully deleted
W20260808 18:27:35.218812 10432 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:41905 (error 108)
I20260808 18:27:35.219751 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 10755
I20260808 18:27:35.226639 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:35.219685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:41905 (local address 127.0.0.1:44180)
0808 18:27:35.226409 (+ 6724us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:35.226529 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6596,"thread_start_us":108,"threads_started":1}
W20260808 18:27:35.226821 10432 catalog_manager.cc:4584] TS 43017a05aa454ed7bfef6c18f744770b (127.7.164.65:41905): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 75dfcf0a4fc74eb1a2b890d542749a62: Network error: Client connection negotiation failed: client connection to 127.7.164.65:41905: connect: Connection refused (error 111)
I20260808 18:27:35.233491 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 10971
I20260808 18:27:35.248093 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 10403
2026-08-08T18:27:35Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6653 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:35 dist-test-slave-jztv krb5kdc[11440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:35 dist-test-slave-jztv krb5kdc[11440](info): set up 2 sockets
Aug 08 18:27:35 dist-test-slave-jztv krb5kdc[11440](info): commencing operation
krb5kdc: starting...
W20260808 18:27:36.497681 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:27:36 dist-test-slave-jztv krb5kdc[11440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:27:36.625605 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:40315
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:40315
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:36.741916 11460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:36.742297 11460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:36.742408 11460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:36.746539 11460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:36.746649 11460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:36.746685 11460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:36.746727 11460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:36.746762 11460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:36.752316 11460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:40315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:40315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:36.753561 11460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:36.754743 11460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:36.764302 11470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:36.764351 11460 server_base.cc:1034] running on GCE node
W20260808 18:27:36.764304 11467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:36.764302 11473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:36.765275 11460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:36.766592 11460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:36.767786 11460 hybrid_clock.cc:648] HybridClock initialized: now 1786213656767763 us; error 36 us; skew 500 ppm
Aug 08 18:27:36 dist-test-slave-jztv krb5kdc[11440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213656, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:36.775350 11460 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:27:36.776369 11460 webserver.cc:460] Webserver started at http://127.7.164.126:38255/ using document root <none> and password file <none>
I20260808 18:27:36.777005 11460 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:36.777132 11460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:36.777415 11460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:36.779775 11460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "e6a3c49faff64d07ab6564853a16d717"
format_stamp: "Formatted at 2026-08-08 18:27:36 on dist-test-slave-jztv"
server_key: "655300f9c9da6864ab35cf2934e3d152"
server_key_iv: "ef94eccf5a0500d0464d491e3a603277"
server_key_version: "encryptionkey@0"
I20260808 18:27:36.780423 11460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "e6a3c49faff64d07ab6564853a16d717"
format_stamp: "Formatted at 2026-08-08 18:27:36 on dist-test-slave-jztv"
server_key: "655300f9c9da6864ab35cf2934e3d152"
server_key_iv: "ef94eccf5a0500d0464d491e3a603277"
server_key_version: "encryptionkey@0"
I20260808 18:27:36.784606 11460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:27:36.787335 11485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:36.788731 11460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:27:36.788900 11460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "e6a3c49faff64d07ab6564853a16d717"
format_stamp: "Formatted at 2026-08-08 18:27:36 on dist-test-slave-jztv"
server_key: "655300f9c9da6864ab35cf2934e3d152"
server_key_iv: "ef94eccf5a0500d0464d491e3a603277"
server_key_version: "encryptionkey@0"
I20260808 18:27:36.789027 11460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:36.802250 11460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:36.805536 11460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:36.805774 11460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:36.813207 11460 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:40315
I20260808 18:27:36.813230 11568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:40315 every 8 connection(s)
I20260808 18:27:36.814195 11460 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:36.817034 11570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:36.822741 11570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717: Bootstrap starting.
I20260808 18:27:36.823372 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 11460
I20260808 18:27:36.823539 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:36.823835 7825 external_mini_cluster.cc:1428] Setting key 4f792ad3e3f0424e811fe5031ec9fb78
I20260808 18:27:36.825533 11570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:36.826339 11570 log.cc:824] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:36.828200 11570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717: No bootstrap required, opened a new log
I20260808 18:27:36.831878 11570 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6a3c49faff64d07ab6564853a16d717" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40315 } }
I20260808 18:27:36.832155 11570 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:36.832227 11570 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6a3c49faff64d07ab6564853a16d717, State: Initialized, Role: FOLLOWER
Aug 08 18:27:36 dist-test-slave-jztv krb5kdc[11440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:36.832739 11570 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6a3c49faff64d07ab6564853a16d717" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40315 } }
I20260808 18:27:36.832913 11570 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:36.832993 11570 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:36.833117 11570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:36.834326 11570 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6a3c49faff64d07ab6564853a16d717" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40315 } }
I20260808 18:27:36.834688 11570 leader_election.cc:302] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6a3c49faff64d07ab6564853a16d717; no voters:
I20260808 18:27:36.835057 11570 leader_election.cc:288] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:36.835176 11576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:36.835407 11576 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [term 1 LEADER]: Becoming Leader. State: Replica: e6a3c49faff64d07ab6564853a16d717, State: Running, Role: LEADER
I20260808 18:27:36.835805 11576 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6a3c49faff64d07ab6564853a16d717" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40315 } }
I20260808 18:27:36.836472 11570 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:36.837975 11579 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6a3c49faff64d07ab6564853a16d717. Latest consensus state: current_term: 1 leader_uuid: "e6a3c49faff64d07ab6564853a16d717" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6a3c49faff64d07ab6564853a16d717" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40315 } } }
I20260808 18:27:36.838042 11577 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6a3c49faff64d07ab6564853a16d717" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6a3c49faff64d07ab6564853a16d717" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40315 } } }
I20260808 18:27:36.838119 11579 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:36.838158 11577 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:36.839761 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:36.825141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46580 (local address 127.7.164.126:40315)
0808 18:27:36.825594 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:36.825610 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:36.825672 (+ 62us) server_negotiation.cc:407] Connection header received
0808 18:27:36.826351 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:36.826369 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:36.826759 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:36.827100 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:36.827936 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:36.828970 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:36.829841 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:36.830394 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:36.830570 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:36.833380 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:36.833385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:36.833415 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:36.833442 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:36.835483 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:36.836040 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:36.836046 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:36.836052 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:36.836192 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:36.836598 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:36.836602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:36.836604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:36.837157 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:36.838149 (+ 992us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:36.838354 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 18:27:36.838667 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":146,"threads_started":1}
W20260808 18:27:36.842747 11598 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:27:36.842842 11598 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:27:36.842911 11595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:36.843632 11595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:36.849959 11595 catalog_manager.cc:1347] Generated new cluster ID: c495b02c47024178920baaa503106de5
I20260808 18:27:36.850033 11595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:36.861920 11595 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:36.863241 11595 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:36.873728 11595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6a3c49faff64d07ab6564853a16d717: Generated new TSK 0
I20260808 18:27:36.874550 11595 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:27:36.911727 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:40315
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:37.046804 11604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:37.047098 11604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:37.047179 11604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:37.051352 11604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:37.051426 11604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:37.051518 11604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:27:37.057072 11604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:40315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:37.058308 11604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:37.059437 11604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:37.069437 11616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:37.069465 11614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:37.069665 11604 server_base.cc:1034] running on GCE node
W20260808 18:27:37.069847 11613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:37.070355 11604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:37.070991 11604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:37.072166 11604 hybrid_clock.cc:648] HybridClock initialized: now 1786213657072150 us; error 28 us; skew 500 ppm
Aug 08 18:27:37 dist-test-slave-jztv krb5kdc[11440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213657, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:37.075138 11604 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:27:37.076134 11604 webserver.cc:460] Webserver started at http://127.7.164.65:45271/ using document root <none> and password file <none>
I20260808 18:27:37.076738 11604 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:37.076833 11604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:37.077077 11604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:37.079275 11604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "839f06c966b846488c4ca372e49a8ad6"
format_stamp: "Formatted at 2026-08-08 18:27:37 on dist-test-slave-jztv"
server_key: "952a8392c253a53667b16ae90907f55c"
server_key_iv: "0b94b8cbfbcd74b57b6e9377410abd2c"
server_key_version: "encryptionkey@0"
I20260808 18:27:37.079923 11604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "839f06c966b846488c4ca372e49a8ad6"
format_stamp: "Formatted at 2026-08-08 18:27:37 on dist-test-slave-jztv"
server_key: "952a8392c253a53667b16ae90907f55c"
server_key_iv: "0b94b8cbfbcd74b57b6e9377410abd2c"
server_key_version: "encryptionkey@0"
I20260808 18:27:37.083905 11604 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:27:37.086793 11631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:37.088204 11604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:27:37.088368 11604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "839f06c966b846488c4ca372e49a8ad6"
format_stamp: "Formatted at 2026-08-08 18:27:37 on dist-test-slave-jztv"
server_key: "952a8392c253a53667b16ae90907f55c"
server_key_iv: "0b94b8cbfbcd74b57b6e9377410abd2c"
server_key_version: "encryptionkey@0"
I20260808 18:27:37.088512 11604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:37.121359 11604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:37.124737 11604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:37.124989 11604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:37.125653 11604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:37.126638 11604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:37.126716 11604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:37.126804 11604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:37.126852 11604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:37.136467 11604 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:33083
I20260808 18:27:37.136498 11809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:33083 every 8 connection(s)
I20260808 18:27:37.137530 11604 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:27:37.140612 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 11604
I20260808 18:27:37.140720 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:27:37.141042 7825 external_mini_cluster.cc:1428] Setting key bf00a9b8e8798f1c4d9b40c3232ddf76
Aug 08 18:27:37 dist-test-slave-jztv krb5kdc[11440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213657, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:27:37.154373 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:37.139532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:58975 (local address 127.7.164.126:40315)
0808 18:27:37.139693 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:37.139697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:37.141890 (+ 2193us) server_negotiation.cc:407] Connection header received
0808 18:27:37.143067 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:37.143071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:37.143131 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:37.143231 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:37.144456 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:37.145143 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:37.146782 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:37.147220 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:37.147350 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:37.150186 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:37.150193 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:37.150199 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:37.150253 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:37.152094 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:37.152767 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:37.152770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:37.152771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:37.152855 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:37.153237 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:37.153240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:37.153242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:37.153439 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:37.153519 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:37.154019 (+ 500us) server_negotiation.cc:299] Negotiation successful
0808 18:27:37.154139 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
I20260808 18:27:37.155330 11813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:37.139997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:40315 (local address 127.7.164.65:58975)
0808 18:27:37.141704 (+ 1707us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:37.141759 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:37.142827 (+ 1068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:37.143402 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:37.143409 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:37.143778 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:37.144290 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:37.144304 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:37.145289 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:37.145295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:37.146646 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:37.146658 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:37.147444 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:37.147448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:37.147623 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:37.148404 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:37.148422 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:37.149980 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:37.152234 (+ 2254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:37.152241 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:37.152265 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:37.152646 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:37.152945 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:37.152947 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:37.152949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:37.153091 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:37.153539 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:37.153545 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:37.153801 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 18:27:37.153969 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 18:27:37.154327 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1449,"thread_start_us":157,"threads_started":1}
I20260808 18:27:37.156702 11810 heartbeater.cc:347] Connected to a master server at 127.7.164.126:40315
I20260808 18:27:37.156976 11810 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:37.157640 11810 heartbeater.cc:511] Master 127.7.164.126:40315 requested a full tablet report, sending...
I20260808 18:27:37.159443 11511 ts_manager.cc:194] Registered new tserver with Master: 839f06c966b846488c4ca372e49a8ad6 (127.7.164.65:33083)
I20260808 18:27:37.160684 11511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:58975
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:27:37.173540 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:40315
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:37.285204 11819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:37.285493 11819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:37.285573 11819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:37.289637 11819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:37.289721 11819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:37.289808 11819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:27:37.295223 11819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:40315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:37.296239 11819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:37.297236 11819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:37.306731 11828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:37.306740 11827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:37.306877 11819 server_base.cc:1034] running on GCE node
W20260808 18:27:37.307106 11832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:37.307503 11819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:37.308120 11819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:37.309312 11819 hybrid_clock.cc:648] HybridClock initialized: now 1786213657309285 us; error 43 us; skew 500 ppm
Aug 08 18:27:37 dist-test-slave-jztv krb5kdc[11440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213657, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:37.312065 11819 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:27:37.312855 11819 webserver.cc:460] Webserver started at http://127.7.164.66:39549/ using document root <none> and password file <none>
I20260808 18:27:37.313349 11819 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:37.313413 11819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:37.313575 11819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:37.315608 11819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "6180e627fd3042ffa9c07ac7dc7f2643"
format_stamp: "Formatted at 2026-08-08 18:27:37 on dist-test-slave-jztv"
server_key: "5558938b5ed51caf9e715447816153f0"
server_key_iv: "8862c2f01bd853d5533f86342d3115e9"
server_key_version: "encryptionkey@0"
I20260808 18:27:37.316250 11819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "6180e627fd3042ffa9c07ac7dc7f2643"
format_stamp: "Formatted at 2026-08-08 18:27:37 on dist-test-slave-jztv"
server_key: "5558938b5ed51caf9e715447816153f0"
server_key_iv: "8862c2f01bd853d5533f86342d3115e9"
server_key_version: "encryptionkey@0"
I20260808 18:27:37.320235 11819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:27:37.322562 11840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:37.323745 11819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:37.323879 11819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "6180e627fd3042ffa9c07ac7dc7f2643"
format_stamp: "Formatted at 2026-08-08 18:27:37 on dist-test-slave-jztv"
server_key: "5558938b5ed51caf9e715447816153f0"
server_key_iv: "8862c2f01bd853d5533f86342d3115e9"
server_key_version: "encryptionkey@0"
I20260808 18:27:37.323982 11819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:37.349022 11819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:37.352034 11819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:37.352241 11819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:37.352850 11819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:37.353677 11819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:37.353724 11819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:37.353792 11819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:37.353830 11819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:37.362679 11819 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:34095
I20260808 18:27:37.362710 12011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:34095 every 8 connection(s)
I20260808 18:27:37.363555 11819 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:27:37.370673 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 11819
I20260808 18:27:37.370759 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:27:37.371017 7825 external_mini_cluster.cc:1428] Setting key 7f72b9a174ff3685b45b7e6dab4b79da
Aug 08 18:27:37 dist-test-slave-jztv krb5kdc[11440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213657, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:37.377211 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:37.365301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:48509 (local address 127.7.164.126:40315)
0808 18:27:37.365437 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:37.365441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:37.366468 (+ 1027us) server_negotiation.cc:407] Connection header received
0808 18:27:37.367294 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:37.367297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:37.367344 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:37.367434 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:37.368503 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:37.369024 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:37.370078 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:37.370307 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:37.370382 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:37.373009 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:37.373013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:37.373035 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:37.373066 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:37.374884 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:37.375469 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:37.375474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:37.375478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:37.375578 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:37.375896 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:37.375899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:37.375901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:37.376142 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:37.376231 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:37.376894 (+ 663us) server_negotiation.cc:299] Negotiation successful
0808 18:27:37.377010 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:27:37.378361 12014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:37.365667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:40315 (local address 127.7.164.66:48509)
0808 18:27:37.366349 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:37.366390 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:37.367121 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:37.367574 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:37.367581 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:37.367910 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:37.368373 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:37.368386 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:37.369132 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:37.369135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:37.369978 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:37.369986 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:37.370431 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:37.370434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:37.370581 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:37.371180 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:37.371196 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:37.372834 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:37.375001 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:37.375005 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:37.375017 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:37.375363 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:37.375673 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:37.375675 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:37.375677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:37.375802 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:37.376263 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:37.376272 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:37.376597 (+ 325us) client_negotiation.cc:769] Sending connection context
0808 18:27:37.376818 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 18:27:37.377170 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":109,"threads_started":1}
I20260808 18:27:37.379633 12012 heartbeater.cc:347] Connected to a master server at 127.7.164.126:40315
I20260808 18:27:37.379889 12012 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:37.380383 12012 heartbeater.cc:511] Master 127.7.164.126:40315 requested a full tablet report, sending...
I20260808 18:27:37.381495 11511 ts_manager.cc:194] Registered new tserver with Master: 6180e627fd3042ffa9c07ac7dc7f2643 (127.7.164.66:34095)
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
I20260808 18:27:37.382364 11511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:48509
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:27:37.398967 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:40315
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:37.511559 12017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:37.511806 12017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:37.511848 12017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:37.515947 12017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:37.516005 12017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:37.516106 12017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:27:37.521723 12017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:40315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:37.522953 12017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:37.524014 12017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:27:37.533876 12017 server_base.cc:1034] running on GCE node
W20260808 18:27:37.533845 12025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:37.533919 12024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:37.534029 12027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:37.534585 12017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:37.535229 12017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:37.536420 12017 hybrid_clock.cc:648] HybridClock initialized: now 1786213657536403 us; error 26 us; skew 500 ppm
Aug 08 18:27:37 dist-test-slave-jztv krb5kdc[11440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213657, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:37.539407 12017 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:27:37.540203 12017 webserver.cc:460] Webserver started at http://127.7.164.67:32879/ using document root <none> and password file <none>
I20260808 18:27:37.540705 12017 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:37.540762 12017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:37.540932 12017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:37.542966 12017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "687463a27419487b9bd236bbc9a83e48"
format_stamp: "Formatted at 2026-08-08 18:27:37 on dist-test-slave-jztv"
server_key: "d9ef29931135b84078d05b103c148c7d"
server_key_iv: "4d4bb690cf4ff6992794359d80d699dd"
server_key_version: "encryptionkey@0"
I20260808 18:27:37.543478 12017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "687463a27419487b9bd236bbc9a83e48"
format_stamp: "Formatted at 2026-08-08 18:27:37 on dist-test-slave-jztv"
server_key: "d9ef29931135b84078d05b103c148c7d"
server_key_iv: "4d4bb690cf4ff6992794359d80d699dd"
server_key_version: "encryptionkey@0"
I20260808 18:27:37.547282 12017 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 18:27:37.549697 12040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:37.550957 12017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:27:37.551054 12017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "687463a27419487b9bd236bbc9a83e48"
format_stamp: "Formatted at 2026-08-08 18:27:37 on dist-test-slave-jztv"
server_key: "d9ef29931135b84078d05b103c148c7d"
server_key_iv: "4d4bb690cf4ff6992794359d80d699dd"
server_key_version: "encryptionkey@0"
I20260808 18:27:37.551137 12017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:37.569518 12017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:37.573014 12017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:37.573177 12017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:37.573734 12017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:37.574638 12017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:37.574689 12017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:37.574731 12017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:37.574752 12017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:37.583781 12017 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:45705
I20260808 18:27:37.583807 12215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:45705 every 8 connection(s)
I20260808 18:27:37.584717 12017 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:27:37.585685 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 12017
I20260808 18:27:37.585810 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:27:37.586122 7825 external_mini_cluster.cc:1428] Setting key f3c503b93b1f926a52fa713a163ea657
Aug 08 18:27:37 dist-test-slave-jztv krb5kdc[11440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213657, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:37.599633 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:37.587430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:40637 (local address 127.7.164.126:40315)
0808 18:27:37.587604 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:37.587610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:37.588868 (+ 1258us) server_negotiation.cc:407] Connection header received
0808 18:27:37.589759 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:37.589763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:37.589816 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:37.589897 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:37.591115 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:37.591759 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:37.593011 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:37.593242 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:37.593351 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:37.595871 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:37.595874 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:37.595876 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:37.595903 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:37.597334 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:37.597817 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:37.597820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:37.597821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:37.597901 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:37.598367 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:37.598370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:37.598371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:37.598587 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:37.598680 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:37.599289 (+ 609us) server_negotiation.cc:299] Negotiation successful
0808 18:27:37.599418 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 18:27:37.600684 12219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:37.587971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:40315 (local address 127.7.164.67:40637)
0808 18:27:37.588729 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:37.588770 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:37.589566 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:37.590069 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:37.590076 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:37.590490 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:37.590970 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:37.590984 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:37.591889 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:37.591893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:37.592883 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:37.592894 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:37.593392 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:37.593396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:37.593567 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:37.594203 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:37.594269 (+ 66us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:37.595711 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:37.597454 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:37.597459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:37.597470 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:37.597720 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:37.598015 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:37.598018 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:37.598020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:37.598164 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:37.598727 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:37.598734 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:37.599038 (+ 304us) client_negotiation.cc:769] Sending connection context
0808 18:27:37.599203 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 18:27:37.599619 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":131,"threads_started":1}
I20260808 18:27:37.601732 12217 heartbeater.cc:347] Connected to a master server at 127.7.164.126:40315
I20260808 18:27:37.601998 12217 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:37.602631 12217 heartbeater.cc:511] Master 127.7.164.126:40315 requested a full tablet report, sending...
I20260808 18:27:37.603641 11511 ts_manager.cc:194] Registered new tserver with Master: 687463a27419487b9bd236bbc9a83e48 (127.7.164.67:45705)
I20260808 18:27:37.604221 11511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:40637
I20260808 18:27:37.612262 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 18:27:37.627452 11574 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:37.625753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46582 (local address 127.7.164.126:40315)
0808 18:27:37.625928 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:37.625933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:37.626082 (+ 149us) server_negotiation.cc:407] Connection header received
0808 18:27:37.626278 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:37.626283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:37.626338 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:37.626437 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:37.627282 (+ 845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46582: BlockingRecv error: recv got EOF from 127.0.0.1:46582 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 18:27:37.628741 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 11604
I20260808 18:27:37.651774 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 11819
I20260808 18:27:37.664119 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 12017
I20260808 18:27:37.676292 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 11460
2026-08-08T18:27:37Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2426 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:37 dist-test-slave-jztv krb5kdc[12232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:37 dist-test-slave-jztv krb5kdc[12232](info): set up 2 sockets
Aug 08 18:27:37 dist-test-slave-jztv krb5kdc[12232](info): commencing operation
krb5kdc: starting...
W20260808 18:27:38.902474 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:27:38 dist-test-slave-jztv krb5kdc[12232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:27:39.037739 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:32891
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:41863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:32891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:39.153443 12252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:39.153712 12252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:39.153752 12252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:39.157752 12252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:39.157819 12252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:39.157837 12252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:39.157853 12252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:39.157868 12252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:39.163118 12252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:41863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:32891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:32891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:39.164120 12252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:39.165129 12252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:39.173745 12260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:39.173766 12262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:39.173777 12264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:39.174062 12252 server_base.cc:1034] running on GCE node
I20260808 18:27:39.174671 12252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:39.175774 12252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:39.176937 12252 hybrid_clock.cc:648] HybridClock initialized: now 1786213659176903 us; error 46 us; skew 500 ppm
Aug 08 18:27:39 dist-test-slave-jztv krb5kdc[12232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213659, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:39.179719 12252 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:27:39.180641 12252 webserver.cc:460] Webserver started at http://127.7.164.126:40009/ using document root <none> and password file <none>
I20260808 18:27:39.181202 12252 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:39.181294 12252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:39.181536 12252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:39.183635 12252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "ee0fd0df6d4642949e9d7036164a5c2f"
format_stamp: "Formatted at 2026-08-08 18:27:39 on dist-test-slave-jztv"
server_key: "03ca50e804a9c697fbaee8aa21c578be"
server_key_iv: "687965035f136882e98e23a05eeb88a8"
server_key_version: "encryptionkey@0"
I20260808 18:27:39.184212 12252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "ee0fd0df6d4642949e9d7036164a5c2f"
format_stamp: "Formatted at 2026-08-08 18:27:39 on dist-test-slave-jztv"
server_key: "03ca50e804a9c697fbaee8aa21c578be"
server_key_iv: "687965035f136882e98e23a05eeb88a8"
server_key_version: "encryptionkey@0"
I20260808 18:27:39.187923 12252 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 18:27:39.190377 12274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:39.191593 12252 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:27:39.191751 12252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "ee0fd0df6d4642949e9d7036164a5c2f"
format_stamp: "Formatted at 2026-08-08 18:27:39 on dist-test-slave-jztv"
server_key: "03ca50e804a9c697fbaee8aa21c578be"
server_key_iv: "687965035f136882e98e23a05eeb88a8"
server_key_version: "encryptionkey@0"
I20260808 18:27:39.191869 12252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:39.214323 12252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:39.217487 12252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:39.217736 12252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:39.225306 12252 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:32891
I20260808 18:27:39.225634 12352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:32891 every 8 connection(s)
I20260808 18:27:39.226403 12252 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:39.229254 12353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:39.234861 12353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f: Bootstrap starting.
I20260808 18:27:39.235688 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 12252
I20260808 18:27:39.235826 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:39.236099 7825 external_mini_cluster.cc:1428] Setting key 29e07ac22e83ecbdd184c2800bef5294
I20260808 18:27:39.237587 12353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:39.238533 12353 log.cc:824] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:39.240806 12353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f: No bootstrap required, opened a new log
Aug 08 18:27:39 dist-test-slave-jztv krb5kdc[12232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:39.245335 12353 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0fd0df6d4642949e9d7036164a5c2f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32891 } }
I20260808 18:27:39.245605 12353 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:39.245694 12353 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee0fd0df6d4642949e9d7036164a5c2f, State: Initialized, Role: FOLLOWER
I20260808 18:27:39.246201 12353 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0fd0df6d4642949e9d7036164a5c2f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32891 } }
I20260808 18:27:39.246398 12353 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:39.246479 12353 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:39.246594 12353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:39.247934 12353 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0fd0df6d4642949e9d7036164a5c2f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32891 } }
I20260808 18:27:39.248333 12353 leader_election.cc:302] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee0fd0df6d4642949e9d7036164a5c2f; no voters:
I20260808 18:27:39.248728 12353 leader_election.cc:288] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:39.248798 12361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:39.249006 12361 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [term 1 LEADER]: Becoming Leader. State: Replica: ee0fd0df6d4642949e9d7036164a5c2f, State: Running, Role: LEADER
I20260808 18:27:39.249401 12361 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0fd0df6d4642949e9d7036164a5c2f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32891 } }
I20260808 18:27:39.250061 12353 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:39.251109 12363 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee0fd0df6d4642949e9d7036164a5c2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0fd0df6d4642949e9d7036164a5c2f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32891 } } }
I20260808 18:27:39.251274 12363 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:39.251675 12377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:39.251747 12365 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee0fd0df6d4642949e9d7036164a5c2f. Latest consensus state: current_term: 1 leader_uuid: "ee0fd0df6d4642949e9d7036164a5c2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee0fd0df6d4642949e9d7036164a5c2f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32891 } } }
I20260808 18:27:39.251884 12358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:39.237229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51970 (local address 127.7.164.126:32891)
0808 18:27:39.237718 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:39.237745 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:39.237791 (+ 46us) server_negotiation.cc:407] Connection header received
0808 18:27:39.238674 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:39.238700 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:39.239109 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:39.239492 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:39.240163 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:39.241655 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:39.242576 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:39.243282 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:39.243454 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:39.246263 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:39.246270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:39.246286 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:39.246319 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:39.248410 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:39.248966 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:39.248972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:39.248978 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:39.249154 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:39.249494 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:39.249498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:39.249500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:39.249935 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:39.250181 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:39.250469 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 18:27:39.250801 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":125,"threads_started":1}
I20260808 18:27:39.252427 12365 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:39.255014 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:27:39.255146 12377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:39.260461 12377 catalog_manager.cc:1347] Generated new cluster ID: dbd9c748d754413cab1993c3ab6a2818
I20260808 18:27:39.260527 12377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:39.280717 12377 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:39.282044 12377 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:39.284670 12358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:39.278718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51982 (local address 127.7.164.126:32891)
0808 18:27:39.278854 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:39.278860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:39.279142 (+ 282us) server_negotiation.cc:407] Connection header received
0808 18:27:39.279290 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:39.279294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:39.279363 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:39.279465 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:39.280132 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:39.280845 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:39.282021 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:39.282436 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:39.282549 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:39.283564 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:39.283568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:39.283571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:39.284169 (+ 598us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:39.284207 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:39.284228 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:39.284323 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:39.284430 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 18:27:39.284517 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:27:39.288065 12377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee0fd0df6d4642949e9d7036164a5c2f: Generated new TSK 0
I20260808 18:27:39.288744 12377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:27:39.293864 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 12252
2026-08-08T18:27:39Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1617 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:39 dist-test-slave-jztv krb5kdc[12404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:39 dist-test-slave-jztv krb5kdc[12404](info): set up 2 sockets
Aug 08 18:27:39 dist-test-slave-jztv krb5kdc[12404](info): commencing operation
krb5kdc: starting...
W20260808 18:27:40.560926 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:27:40 dist-test-slave-jztv krb5kdc[12404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:27:40.697522 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:35059
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:35059
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:40.811576 12419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:40.811877 12419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:40.811950 12419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:40.816040 12419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:40.816135 12419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:40.816172 12419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:40.816212 12419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:40.816248 12419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:40.821656 12419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:35059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:35059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:40.822892 12419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:40.823956 12419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:40.832916 12428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:40.832924 12426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:40.832924 12425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:40.832978 12419 server_base.cc:1034] running on GCE node
I20260808 18:27:40.833623 12419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:40.834810 12419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:40.836011 12419 hybrid_clock.cc:648] HybridClock initialized: now 1786213660835998 us; error 30 us; skew 500 ppm
Aug 08 18:27:40 dist-test-slave-jztv krb5kdc[12404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213660, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:40.838713 12419 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:27:40.839587 12419 webserver.cc:460] Webserver started at http://127.7.164.126:36577/ using document root <none> and password file <none>
I20260808 18:27:40.840152 12419 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:40.840245 12419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:40.840479 12419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:40.842573 12419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "8ad99d7ef01044d5b83c01fed99abebc"
format_stamp: "Formatted at 2026-08-08 18:27:40 on dist-test-slave-jztv"
server_key: "18ab7114c7126beb0a7ea482e75d3472"
server_key_iv: "0edaa600841987f0868fc7c7c0e3fa2a"
server_key_version: "encryptionkey@0"
I20260808 18:27:40.843144 12419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "8ad99d7ef01044d5b83c01fed99abebc"
format_stamp: "Formatted at 2026-08-08 18:27:40 on dist-test-slave-jztv"
server_key: "18ab7114c7126beb0a7ea482e75d3472"
server_key_iv: "0edaa600841987f0868fc7c7c0e3fa2a"
server_key_version: "encryptionkey@0"
I20260808 18:27:40.846902 12419 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:27:40.849229 12440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:40.850472 12419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 18:27:40.850606 12419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "8ad99d7ef01044d5b83c01fed99abebc"
format_stamp: "Formatted at 2026-08-08 18:27:40 on dist-test-slave-jztv"
server_key: "18ab7114c7126beb0a7ea482e75d3472"
server_key_iv: "0edaa600841987f0868fc7c7c0e3fa2a"
server_key_version: "encryptionkey@0"
I20260808 18:27:40.850714 12419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:40.871803 12419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:40.874696 12419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:40.874853 12419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:40.881608 12419 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:35059
I20260808 18:27:40.881896 12528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:35059 every 8 connection(s)
I20260808 18:27:40.882668 12419 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:40.884756 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 12419
I20260808 18:27:40.884853 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:40.885171 7825 external_mini_cluster.cc:1428] Setting key 32815b3eed3841c120548ea8cd771e58
I20260808 18:27:40.885743 12530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:40.892975 12530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc: Bootstrap starting.
Aug 08 18:27:40 dist-test-slave-jztv krb5kdc[12404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:40.895665 12530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:40.896585 12530 log.cc:824] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:40.898775 12530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc: No bootstrap required, opened a new log
I20260808 18:27:40.900333 12534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:40.886631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41930 (local address 127.7.164.126:35059)
0808 18:27:40.887056 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:40.887076 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:40.887118 (+ 42us) server_negotiation.cc:407] Connection header received
0808 18:27:40.887973 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:40.887996 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:40.888412 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:40.888796 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:40.889530 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:40.891043 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:40.891884 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:40.892576 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:40.892731 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:40.895443 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:40.895449 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:40.895461 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:40.895488 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:40.897208 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:40.897667 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:40.897673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:40.897678 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:40.897810 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:40.898103 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:40.898106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:40.898108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:40.898649 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:40.898851 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:40.899119 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 18:27:40.899433 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":111,"threads_started":1}
I20260808 18:27:40.903118 12530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad99d7ef01044d5b83c01fed99abebc" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35059 } }
I20260808 18:27:40.903326 12530 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:40.903362 12530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ad99d7ef01044d5b83c01fed99abebc, State: Initialized, Role: FOLLOWER
I20260808 18:27:40.903715 12530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad99d7ef01044d5b83c01fed99abebc" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35059 } }
I20260808 18:27:40.903824 12530 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:40.903870 12530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:40.903939 12530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:40.904788 12530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad99d7ef01044d5b83c01fed99abebc" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35059 } }
I20260808 18:27:40.905082 12530 leader_election.cc:302] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ad99d7ef01044d5b83c01fed99abebc; no voters:
I20260808 18:27:40.905351 12530 leader_election.cc:288] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:40.905490 12536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:40.905733 12536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [term 1 LEADER]: Becoming Leader. State: Replica: 8ad99d7ef01044d5b83c01fed99abebc, State: Running, Role: LEADER
I20260808 18:27:40.906119 12536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad99d7ef01044d5b83c01fed99abebc" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35059 } }
I20260808 18:27:40.906587 12530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:40.907812 12539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ad99d7ef01044d5b83c01fed99abebc. Latest consensus state: current_term: 1 leader_uuid: "8ad99d7ef01044d5b83c01fed99abebc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad99d7ef01044d5b83c01fed99abebc" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35059 } } }
I20260808 18:27:40.907953 12539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:40.907954 12537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ad99d7ef01044d5b83c01fed99abebc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad99d7ef01044d5b83c01fed99abebc" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35059 } } }
I20260808 18:27:40.908082 12537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:40.908460 12549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:40.911382 12549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:40.917098 12549 catalog_manager.cc:1347] Generated new cluster ID: 8d2c812a9e944ef9ad6da821d352783e
I20260808 18:27:40.917169 12549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:40.927188 12549 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:40.928094 12549 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:40.936161 12549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ad99d7ef01044d5b83c01fed99abebc: Generated new TSK 0
I20260808 18:27:40.936866 12549 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:27:40.974304 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35059
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:41.091825 12568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:41.092113 12568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:41.092159 12568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:41.096266 12568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:41.096342 12568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:41.096403 12568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:27:41.102007 12568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:41.103235 12568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:41.104308 12568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:41.114622 12573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:41.114622 12576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:41.114634 12574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:41.114734 12568 server_base.cc:1034] running on GCE node
I20260808 18:27:41.115440 12568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:41.116185 12568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:41.117369 12568 hybrid_clock.cc:648] HybridClock initialized: now 1786213661117339 us; error 39 us; skew 500 ppm
Aug 08 18:27:41 dist-test-slave-jztv krb5kdc[12404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213661, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:41.120381 12568 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:27:41.121372 12568 webserver.cc:460] Webserver started at http://127.7.164.65:32941/ using document root <none> and password file <none>
I20260808 18:27:41.121982 12568 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:41.122051 12568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:41.122345 12568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:41.124545 12568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "3520afcc76b6452f99cccb43057c1414"
format_stamp: "Formatted at 2026-08-08 18:27:41 on dist-test-slave-jztv"
server_key: "cb9a33199ffec55a5575d21803ba6086"
server_key_iv: "5aed7f6d35d6559a6d0608f37614fd77"
server_key_version: "encryptionkey@0"
I20260808 18:27:41.125103 12568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "3520afcc76b6452f99cccb43057c1414"
format_stamp: "Formatted at 2026-08-08 18:27:41 on dist-test-slave-jztv"
server_key: "cb9a33199ffec55a5575d21803ba6086"
server_key_iv: "5aed7f6d35d6559a6d0608f37614fd77"
server_key_version: "encryptionkey@0"
I20260808 18:27:41.129196 12568 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 18:27:41.131939 12585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:41.133352 12568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 18:27:41.133505 12568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "3520afcc76b6452f99cccb43057c1414"
format_stamp: "Formatted at 2026-08-08 18:27:41 on dist-test-slave-jztv"
server_key: "cb9a33199ffec55a5575d21803ba6086"
server_key_iv: "5aed7f6d35d6559a6d0608f37614fd77"
server_key_version: "encryptionkey@0"
I20260808 18:27:41.133639 12568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:41.170848 12568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:41.174322 12568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:41.174530 12568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:41.175168 12568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:41.176047 12568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:41.176119 12568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:41.176192 12568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:41.176236 12568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:41.185154 12568 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:40029
I20260808 18:27:41.185178 12756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:40029 every 8 connection(s)
I20260808 18:27:41.186055 12568 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:27:41.192483 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 12568
I20260808 18:27:41.192589 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:27:41.192867 7825 external_mini_cluster.cc:1428] Setting key e1b01933b5d4ef707f5ff83229904aac
Aug 08 18:27:41 dist-test-slave-jztv krb5kdc[12404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213661, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:41.201329 12534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:41.187957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:57303 (local address 127.7.164.126:35059)
0808 18:27:41.188111 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:41.188116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:41.189004 (+ 888us) server_negotiation.cc:407] Connection header received
0808 18:27:41.190387 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:41.190390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:41.190439 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:41.190513 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:41.191678 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:41.192267 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:41.193629 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:41.193991 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:41.194098 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:41.196948 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:41.196951 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:41.196953 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:41.196979 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:41.198831 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:41.199425 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:41.199432 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:41.199437 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:41.199564 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:41.199991 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:41.199998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:41.200000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:41.200266 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:41.200385 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:41.200903 (+ 518us) server_negotiation.cc:299] Negotiation successful
0808 18:27:41.201065 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":57}
I20260808 18:27:41.202014 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:41.188388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35059 (local address 127.7.164.65:57303)
0808 18:27:41.188849 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:41.188893 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:41.190129 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:41.190683 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:41.190690 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:41.191044 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:41.191535 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:41.191549 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:41.192405 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:41.192408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:41.193485 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:41.193495 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:41.194189 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:41.194196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:41.194420 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:41.195059 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:41.195085 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:41.196747 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:41.198977 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:41.198983 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:41.198997 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:41.199298 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:41.199695 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:41.199700 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:41.199702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:41.199883 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:41.200380 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:41.200386 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:41.200645 (+ 259us) client_negotiation.cc:769] Sending connection context
0808 18:27:41.200810 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 18:27:41.201091 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":109,"threads_started":1}
I20260808 18:27:41.203414 12760 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35059
I20260808 18:27:41.203683 12760 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:27:41.204239 12760 heartbeater.cc:511] Master 127.7.164.126:35059 requested a full tablet report, sending...
I20260808 18:27:41.205857 12456 ts_manager.cc:194] Registered new tserver with Master: 3520afcc76b6452f99cccb43057c1414 (127.7.164.65:40029)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
I20260808 18:27:41.207372 12456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:57303
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:27:41.223554 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35059
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:41.338738 12767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:41.339007 12767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:41.339058 12767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:41.343132 12767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:41.343185 12767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:41.343243 12767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:27:41.348373 12767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:41.349386 12767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:41.350454 12767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:41.359859 12783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:41.359897 12779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:41.360075 12767 server_base.cc:1034] running on GCE node
W20260808 18:27:41.359931 12778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:41.360510 12767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:41.361204 12767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:41.362381 12767 hybrid_clock.cc:648] HybridClock initialized: now 1786213661362363 us; error 26 us; skew 500 ppm
Aug 08 18:27:41 dist-test-slave-jztv krb5kdc[12404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213661, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:41.365298 12767 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:27:41.366341 12767 webserver.cc:460] Webserver started at http://127.7.164.66:44299/ using document root <none> and password file <none>
I20260808 18:27:41.366926 12767 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:41.367022 12767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:41.367270 12767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:41.369365 12767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "ea965d2244764808990ae811ee93be91"
format_stamp: "Formatted at 2026-08-08 18:27:41 on dist-test-slave-jztv"
server_key: "88cb01301fd66a4919f079cbf8cbfeda"
server_key_iv: "0e942eb8198e878af4795f7746adf5f7"
server_key_version: "encryptionkey@0"
I20260808 18:27:41.369963 12767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "ea965d2244764808990ae811ee93be91"
format_stamp: "Formatted at 2026-08-08 18:27:41 on dist-test-slave-jztv"
server_key: "88cb01301fd66a4919f079cbf8cbfeda"
server_key_iv: "0e942eb8198e878af4795f7746adf5f7"
server_key_version: "encryptionkey@0"
I20260808 18:27:41.373896 12767 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:27:41.376652 12792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:41.378005 12767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:27:41.378161 12767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "ea965d2244764808990ae811ee93be91"
format_stamp: "Formatted at 2026-08-08 18:27:41 on dist-test-slave-jztv"
server_key: "88cb01301fd66a4919f079cbf8cbfeda"
server_key_iv: "0e942eb8198e878af4795f7746adf5f7"
server_key_version: "encryptionkey@0"
I20260808 18:27:41.378343 12767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:41.396890 12767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:41.400856 12767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:41.401113 12767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:41.401963 12767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:41.402998 12767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:41.403079 12767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:41.403153 12767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:41.403205 12767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:41.413035 12767 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:40129
I20260808 18:27:41.413064 12960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:40129 every 8 connection(s)
I20260808 18:27:41.413940 12767 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:27:41.421428 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 12767
I20260808 18:27:41.421608 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:27:41.421892 7825 external_mini_cluster.cc:1428] Setting key a2e12b1a35fc406333da53e1d2e1d4f0
Aug 08 18:27:41 dist-test-slave-jztv krb5kdc[12404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213661, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:41.428969 12534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:41.415766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:33237 (local address 127.7.164.126:35059)
0808 18:27:41.415905 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:41.415909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:41.416768 (+ 859us) server_negotiation.cc:407] Connection header received
0808 18:27:41.418196 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:41.418200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:41.418283 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:41.418397 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:41.419620 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:41.420171 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:41.421495 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:41.421857 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:41.421985 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:41.424938 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:41.424947 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:41.424949 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:41.424973 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:41.426559 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:41.427194 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:41.427197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:41.427198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:41.427276 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:41.427686 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:41.427688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:41.427689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:41.427859 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:41.427934 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:41.428704 (+ 770us) server_negotiation.cc:299] Negotiation successful
0808 18:27:41.428842 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:27:41.430323 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:41.416095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35059 (local address 127.7.164.66:33237)
0808 18:27:41.416592 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:41.416638 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:41.417966 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:41.418582 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:41.418589 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:41.418952 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:41.419463 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:41.419477 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:41.420308 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:41.420312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:41.421359 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:41.421372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:41.422031 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:41.422036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:41.422273 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:41.422938 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:41.422955 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:41.424763 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:41.426679 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:41.426687 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:41.426706 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:41.427090 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:41.427369 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:41.427375 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:41.427380 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:41.427590 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:41.427965 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:41.427975 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:41.428369 (+ 394us) client_negotiation.cc:769] Sending connection context
0808 18:27:41.428629 (+ 260us) client_negotiation.cc:240] Negotiation successful
0808 18:27:41.429045 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":114,"threads_started":1}
I20260808 18:27:41.431753 12962 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35059
I20260808 18:27:41.432047 12962 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:41.432614 12962 heartbeater.cc:511] Master 127.7.164.126:35059 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
I20260808 18:27:41.433800 12456 ts_manager.cc:194] Registered new tserver with Master: ea965d2244764808990ae811ee93be91 (127.7.164.66:40129)
I20260808 18:27:41.434585 12456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:33237
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:27:41.453080 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35059
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:41.568032 12972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:41.568325 12972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:41.568401 12972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:41.572443 12972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:41.572531 12972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:41.572630 12972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:27:41.577910 12972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:41.579062 12972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:41.580127 12972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:41.590143 12982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:41.590354 12972 server_base.cc:1034] running on GCE node
W20260808 18:27:41.590111 12981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:41.590116 12984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:41.590969 12972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:41.591604 12972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:41.592818 12972 hybrid_clock.cc:648] HybridClock initialized: now 1786213661592801 us; error 35 us; skew 500 ppm
Aug 08 18:27:41 dist-test-slave-jztv krb5kdc[12404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213661, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:41.595657 12972 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:27:41.596585 12972 webserver.cc:460] Webserver started at http://127.7.164.67:39059/ using document root <none> and password file <none>
I20260808 18:27:41.597179 12972 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:41.597277 12972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:41.597523 12972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:41.599699 12972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "13760d70b55f4b2eaed35217f304c180"
format_stamp: "Formatted at 2026-08-08 18:27:41 on dist-test-slave-jztv"
server_key: "5d50570086b2a576dde95570abcd14fd"
server_key_iv: "3e4cf6d8dc413c52bce7ec3b9234fa04"
server_key_version: "encryptionkey@0"
I20260808 18:27:41.600332 12972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "13760d70b55f4b2eaed35217f304c180"
format_stamp: "Formatted at 2026-08-08 18:27:41 on dist-test-slave-jztv"
server_key: "5d50570086b2a576dde95570abcd14fd"
server_key_iv: "3e4cf6d8dc413c52bce7ec3b9234fa04"
server_key_version: "encryptionkey@0"
I20260808 18:27:41.604202 12972 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:27:41.606719 12993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:41.608155 12972 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:27:41.608287 12972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "13760d70b55f4b2eaed35217f304c180"
format_stamp: "Formatted at 2026-08-08 18:27:41 on dist-test-slave-jztv"
server_key: "5d50570086b2a576dde95570abcd14fd"
server_key_iv: "3e4cf6d8dc413c52bce7ec3b9234fa04"
server_key_version: "encryptionkey@0"
I20260808 18:27:41.608395 12972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:41.633705 12972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:41.637460 12972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:41.637740 12972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:41.638489 12972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:41.639387 12972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:41.639467 12972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:41.639552 12972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:41.639604 12972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:41.648964 12972 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:43247
I20260808 18:27:41.649020 13162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:43247 every 8 connection(s)
I20260808 18:27:41.649911 12972 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:27:41.650909 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 12972
I20260808 18:27:41.651034 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:27:41.651353 7825 external_mini_cluster.cc:1428] Setting key 777a7d2aac988f5cf7c37f5a81e73ed7
Aug 08 18:27:41 dist-test-slave-jztv krb5kdc[12404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213661, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:41.663702 12534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:41.652030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:52701 (local address 127.7.164.126:35059)
0808 18:27:41.652191 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:41.652194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:41.653221 (+ 1027us) server_negotiation.cc:407] Connection header received
0808 18:27:41.654771 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:41.654774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:41.654820 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:41.654894 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:41.656047 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:41.656573 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:41.657675 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:41.657884 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:41.657960 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:41.660369 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:41.660372 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:41.660374 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:41.660399 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:41.661866 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:41.662376 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:41.662378 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:41.662380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:41.662456 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:41.662766 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:41.662768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:41.662769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:41.662943 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:41.663016 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:41.663477 (+ 461us) server_negotiation.cc:299] Negotiation successful
0808 18:27:41.663580 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:27:41.664724 13167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:41.652516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35059 (local address 127.7.164.67:52701)
0808 18:27:41.653076 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:41.653127 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:41.654523 (+ 1396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:41.655074 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:41.655080 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:41.655433 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:41.655918 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:41.655930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:41.656680 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:41.656683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:41.657556 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:41.657565 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:41.658011 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:41.658014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:41.658156 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:41.658756 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:41.658772 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:41.660211 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:41.661965 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:41.661970 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:41.661982 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:41.662297 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:41.662532 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:41.662534 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:41.662535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:41.662677 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:41.663030 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:41.663035 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:41.663266 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 18:27:41.663416 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 18:27:41.663677 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":152,"threads_started":1}
I20260808 18:27:41.666007 13163 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35059
I20260808 18:27:41.666275 13163 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:41.666754 13163 heartbeater.cc:511] Master 127.7.164.126:35059 requested a full tablet report, sending...
I20260808 18:27:41.667738 12456 ts_manager.cc:194] Registered new tserver with Master: 13760d70b55f4b2eaed35217f304c180 (127.7.164.67:43247)
I20260808 18:27:41.668321 12456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:52701
I20260808 18:27:41.677852 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 18:27:41 dist-test-slave-jztv krb5kdc[12404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213661, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 18:27:41 dist-test-slave-jztv krb5kdc[12404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213661, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:41.706434 12534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:41.697733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41940 (local address 127.7.164.126:35059)
0808 18:27:41.697946 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:41.697951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:41.698112 (+ 161us) server_negotiation.cc:407] Connection header received
0808 18:27:41.698362 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:41.698365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:41.698422 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:41.698517 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:41.699288 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:41.699842 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:41.700698 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:41.700932 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:41.701040 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:41.703542 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:41.703545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:41.703547 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:41.703582 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:41.704821 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:41.705208 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:41.705210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:41.705212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:41.705289 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:41.705516 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:41.705518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:41.705519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:41.705745 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:41.705848 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:41.706052 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 18:27:41.706153 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260808 18:27:41.706960 12456 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41940
I20260808 18:27:41.708112 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 12568
I20260808 18:27:41.724514 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 12767
I20260808 18:27:41.736898 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 12972
I20260808 18:27:41.744949 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 12419
2026-08-08T18:27:41Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2449 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:41 dist-test-slave-jztv krb5kdc[13183](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:41 dist-test-slave-jztv krb5kdc[13183](info): set up 2 sockets
Aug 08 18:27:41 dist-test-slave-jztv krb5kdc[13183](info): commencing operation
krb5kdc: starting...
W20260808 18:27:42.975621 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:27:43 dist-test-slave-jztv krb5kdc[13183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:27:43.105427 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:46189
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:46189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:43.219027 13203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:43.219409 13203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:43.219522 13203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:43.223611 13203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:43.223712 13203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:43.223753 13203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:43.223791 13203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:43.223825 13203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:43.229107 13203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:46189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:46189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:43.230244 13203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:43.231281 13203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:43.240307 13214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:43.240342 13210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:43.240381 13212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:43.240474 13203 server_base.cc:1034] running on GCE node
I20260808 18:27:43.241245 13203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:43.242477 13203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:43.243691 13203 hybrid_clock.cc:648] HybridClock initialized: now 1786213663243675 us; error 25 us; skew 500 ppm
Aug 08 18:27:43 dist-test-slave-jztv krb5kdc[13183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213663, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:43.246677 13203 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:27:43.247606 13203 webserver.cc:460] Webserver started at http://127.7.164.126:36469/ using document root <none> and password file <none>
I20260808 18:27:43.248188 13203 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:43.248286 13203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:43.248560 13203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:43.250871 13203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "f97a40fcf5a74c29b619af82cae08eef"
format_stamp: "Formatted at 2026-08-08 18:27:43 on dist-test-slave-jztv"
server_key: "ab271282c79c813f89e43e50361600b1"
server_key_iv: "a223f19cbfbe0376d5366fc7b1ee1e54"
server_key_version: "encryptionkey@0"
I20260808 18:27:43.251474 13203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "f97a40fcf5a74c29b619af82cae08eef"
format_stamp: "Formatted at 2026-08-08 18:27:43 on dist-test-slave-jztv"
server_key: "ab271282c79c813f89e43e50361600b1"
server_key_iv: "a223f19cbfbe0376d5366fc7b1ee1e54"
server_key_version: "encryptionkey@0"
I20260808 18:27:43.255365 13203 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:27:43.257853 13230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:43.259172 13203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:43.259321 13203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "f97a40fcf5a74c29b619af82cae08eef"
format_stamp: "Formatted at 2026-08-08 18:27:43 on dist-test-slave-jztv"
server_key: "ab271282c79c813f89e43e50361600b1"
server_key_iv: "a223f19cbfbe0376d5366fc7b1ee1e54"
server_key_version: "encryptionkey@0"
I20260808 18:27:43.259472 13203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:43.274632 13203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:43.277501 13203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:43.277712 13203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:43.285362 13203 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:46189
I20260808 18:27:43.285431 13308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:46189 every 8 connection(s)
I20260808 18:27:43.286425 13203 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:43.289220 13309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:43.292642 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 13203
I20260808 18:27:43.292788 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:43.293145 7825 external_mini_cluster.cc:1428] Setting key 810d38a8edb6ab15a3ce147a1c3c2a9b
I20260808 18:27:43.295401 13309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef: Bootstrap starting.
I20260808 18:27:43.297996 13309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:43.298890 13309 log.cc:824] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:43.301538 13309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef: No bootstrap required, opened a new log
Aug 08 18:27:43 dist-test-slave-jztv krb5kdc[13183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:43.305930 13309 raft_consensus.cc:348] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97a40fcf5a74c29b619af82cae08eef" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 46189 } }
I20260808 18:27:43.306236 13309 raft_consensus.cc:374] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:43.306304 13309 raft_consensus.cc:729] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97a40fcf5a74c29b619af82cae08eef, State: Initialized, Role: FOLLOWER
I20260808 18:27:43.306844 13309 consensus_queue.cc:260] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97a40fcf5a74c29b619af82cae08eef" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 46189 } }
I20260808 18:27:43.307019 13309 raft_consensus.cc:388] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:43.307117 13309 raft_consensus.cc:482] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:43.307231 13309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:43.308482 13309 raft_consensus.cc:504] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97a40fcf5a74c29b619af82cae08eef" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 46189 } }
I20260808 18:27:43.308853 13309 leader_election.cc:302] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f97a40fcf5a74c29b619af82cae08eef; no voters:
I20260808 18:27:43.309207 13309 leader_election.cc:288] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:43.309329 13315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:43.309655 13315 raft_consensus.cc:686] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [term 1 LEADER]: Becoming Leader. State: Replica: f97a40fcf5a74c29b619af82cae08eef, State: Running, Role: LEADER
I20260808 18:27:43.309973 13315 consensus_queue.cc:237] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97a40fcf5a74c29b619af82cae08eef" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 46189 } }
I20260808 18:27:43.310092 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:43.294672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52580 (local address 127.7.164.126:46189)
0808 18:27:43.295114 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:43.295130 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:43.295167 (+ 37us) server_negotiation.cc:407] Connection header received
0808 18:27:43.295849 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:43.295868 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:43.296206 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:43.296477 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:43.297454 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:43.298651 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:43.299969 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:43.300462 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:43.300611 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:43.303538 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:43.303548 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:43.303568 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:43.303602 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:43.306642 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:43.307157 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:43.307166 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:43.307172 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:43.307328 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:43.307650 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:43.307653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:43.307655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:43.308201 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:43.308421 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:43.308658 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 18:27:43.308987 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":147,"threads_started":1}
I20260808 18:27:43.310564 13309 sys_catalog.cc:564] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:43.312813 13317 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [sys.catalog]: SysCatalogTable state changed. Reason: New leader f97a40fcf5a74c29b619af82cae08eef. Latest consensus state: current_term: 1 leader_uuid: "f97a40fcf5a74c29b619af82cae08eef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97a40fcf5a74c29b619af82cae08eef" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 46189 } } }
I20260808 18:27:43.312909 13318 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f97a40fcf5a74c29b619af82cae08eef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97a40fcf5a74c29b619af82cae08eef" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 46189 } } }
I20260808 18:27:43.313054 13317 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:43.313059 13318 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef [sys.catalog]: This master's current role is: LEADER
W20260808 18:27:43.315747 13334 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:27:43.315848 13334 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:27:43.315963 13335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:43.316856 13335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:43.322176 13335 catalog_manager.cc:1347] Generated new cluster ID: 9b530aa572c846dea6b73ec430bba6a3
I20260808 18:27:43.322285 13335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:43.332830 13335 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:43.333776 13335 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:43.338810 13335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f97a40fcf5a74c29b619af82cae08eef: Generated new TSK 0
I20260808 18:27:43.339504 13335 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:27:43.377156 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:46189
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:43.490880 13344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:43.491142 13344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:43.491216 13344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:43.495324 13344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:43.495388 13344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:43.495477 13344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:27:43.500579 13344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:46189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:43.501608 13344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:43.502633 13344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:27:43.512351 13344 server_base.cc:1034] running on GCE node
W20260808 18:27:43.512295 13352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:43.512373 13349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:43.512302 13350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:43.512928 13344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:43.513506 13344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:43.514707 13344 hybrid_clock.cc:648] HybridClock initialized: now 1786213663514681 us; error 48 us; skew 500 ppm
Aug 08 18:27:43 dist-test-slave-jztv krb5kdc[13183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213663, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:43.517426 13344 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:27:43.518317 13344 webserver.cc:460] Webserver started at http://127.7.164.65:43357/ using document root <none> and password file <none>
I20260808 18:27:43.518874 13344 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:43.518965 13344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:43.519196 13344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:43.521245 13344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "d856116e859445a0836b891b198c31a3"
format_stamp: "Formatted at 2026-08-08 18:27:43 on dist-test-slave-jztv"
server_key: "9c8aa6e95897256bd21248222c3537e4"
server_key_iv: "39bdb8e6bdbcb61abcbd9ad93762b3c6"
server_key_version: "encryptionkey@0"
I20260808 18:27:43.521826 13344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "d856116e859445a0836b891b198c31a3"
format_stamp: "Formatted at 2026-08-08 18:27:43 on dist-test-slave-jztv"
server_key: "9c8aa6e95897256bd21248222c3537e4"
server_key_iv: "39bdb8e6bdbcb61abcbd9ad93762b3c6"
server_key_version: "encryptionkey@0"
I20260808 18:27:43.525466 13344 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:27:43.527946 13367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:43.529194 13344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:27:43.529310 13344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "d856116e859445a0836b891b198c31a3"
format_stamp: "Formatted at 2026-08-08 18:27:43 on dist-test-slave-jztv"
server_key: "9c8aa6e95897256bd21248222c3537e4"
server_key_iv: "39bdb8e6bdbcb61abcbd9ad93762b3c6"
server_key_version: "encryptionkey@0"
I20260808 18:27:43.529419 13344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:43.560593 13344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:43.563740 13344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:43.563910 13344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:43.564525 13344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:43.565362 13344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:43.565408 13344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:43.565447 13344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:43.565485 13344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:43.574596 13344 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:43437
I20260808 18:27:43.574651 13527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:43437 every 8 connection(s)
I20260808 18:27:43.575510 13344 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:27:43.584589 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 13344
I20260808 18:27:43.584705 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:27:43.585071 7825 external_mini_cluster.cc:1428] Setting key b6a08cc372bd0f41f8386208061f1dce
Aug 08 18:27:43 dist-test-slave-jztv krb5kdc[13183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213663, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:43.590514 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:43.577366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:56869 (local address 127.7.164.126:46189)
0808 18:27:43.577510 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:43.577514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:43.578405 (+ 891us) server_negotiation.cc:407] Connection header received
0808 18:27:43.579865 (+ 1460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:43.579869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:43.579933 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:43.580018 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:43.581280 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:43.581843 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:43.583111 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:43.583346 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:43.583462 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:43.586242 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:43.586246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:43.586249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:43.586275 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:43.587848 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:43.588504 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:43.588507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:43.588508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:43.588592 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:43.589073 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:43.589079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:43.589082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:43.589344 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:43.589490 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:43.590068 (+ 578us) server_negotiation.cc:299] Negotiation successful
0808 18:27:43.590259 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:27:43.591223 13531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:43.577755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:46189 (local address 127.7.164.65:56869)
0808 18:27:43.578205 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:43.578299 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:43.579635 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:43.580225 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:43.580232 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:43.580623 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:43.581115 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:43.581129 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:43.581991 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:43.581999 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:43.582958 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:43.582969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:43.583508 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:43.583512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:43.583694 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:43.584284 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:43.584337 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:43.586046 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:43.587976 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:43.587986 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:43.588007 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:43.588389 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:43.588743 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:43.588749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:43.588754 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:43.588958 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:43.589486 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:43.589491 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:43.589811 (+ 320us) client_negotiation.cc:769] Sending connection context
0808 18:27:43.589976 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 18:27:43.590305 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":118,"threads_started":1}
I20260808 18:27:43.592535 13528 heartbeater.cc:347] Connected to a master server at 127.7.164.126:46189
I20260808 18:27:43.592821 13528 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:43.593498 13528 heartbeater.cc:511] Master 127.7.164.126:46189 requested a full tablet report, sending...
I20260808 18:27:43.595211 13251 ts_manager.cc:194] Registered new tserver with Master: d856116e859445a0836b891b198c31a3 (127.7.164.65:43437)
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:27:43.596413 13251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:56869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:27:43.615453 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:46189
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:43.730119 13534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:43.730479 13534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:43.730526 13534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:43.734778 13534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:43.734841 13534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:43.734903 13534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:27:43.740250 13534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:46189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:43.741339 13534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:43.742383 13534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:43.752310 13545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:43.752389 13534 server_base.cc:1034] running on GCE node
W20260808 18:27:43.752310 13541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:43.752310 13547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:43.753021 13534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:43.753671 13534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:43.754870 13534 hybrid_clock.cc:648] HybridClock initialized: now 1786213663754831 us; error 45 us; skew 500 ppm
Aug 08 18:27:43 dist-test-slave-jztv krb5kdc[13183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213663, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:43.757735 13534 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:27:43.758708 13534 webserver.cc:460] Webserver started at http://127.7.164.66:33061/ using document root <none> and password file <none>
I20260808 18:27:43.759296 13534 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:43.759382 13534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:43.759613 13534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:43.761703 13534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "3be45c0ff21e46e0a8cd2fbfcc744381"
format_stamp: "Formatted at 2026-08-08 18:27:43 on dist-test-slave-jztv"
server_key: "20d83962ab656de5cf7bae5b2e469bcd"
server_key_iv: "4805e5c65694812cb84570b8f6ec678f"
server_key_version: "encryptionkey@0"
I20260808 18:27:43.762428 13534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "3be45c0ff21e46e0a8cd2fbfcc744381"
format_stamp: "Formatted at 2026-08-08 18:27:43 on dist-test-slave-jztv"
server_key: "20d83962ab656de5cf7bae5b2e469bcd"
server_key_iv: "4805e5c65694812cb84570b8f6ec678f"
server_key_version: "encryptionkey@0"
I20260808 18:27:43.766185 13534 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:27:43.768692 13558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:43.769903 13534 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:43.770030 13534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "3be45c0ff21e46e0a8cd2fbfcc744381"
format_stamp: "Formatted at 2026-08-08 18:27:43 on dist-test-slave-jztv"
server_key: "20d83962ab656de5cf7bae5b2e469bcd"
server_key_iv: "4805e5c65694812cb84570b8f6ec678f"
server_key_version: "encryptionkey@0"
I20260808 18:27:43.770133 13534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:43.786053 13534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:43.788719 13534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:43.788915 13534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:43.789495 13534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:43.790380 13534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:43.790429 13534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:43.790498 13534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:43.790536 13534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:43.799475 13534 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:37035
I20260808 18:27:43.799485 13730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:37035 every 8 connection(s)
I20260808 18:27:43.800374 13534 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:27:43.802887 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 13534
I20260808 18:27:43.803018 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:27:43.803395 7825 external_mini_cluster.cc:1428] Setting key 0af21348814f47cfe5518471046cb1e7
Aug 08 18:27:43 dist-test-slave-jztv krb5kdc[13183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213663, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
I20260808 18:27:43.815864 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:43.802148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:43507 (local address 127.7.164.126:46189)
0808 18:27:43.802341 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:43.802346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:43.803289 (+ 943us) server_negotiation.cc:407] Connection header received
0808 18:27:43.804997 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:43.805000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:43.805043 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:43.805113 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:43.806739 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:43.807298 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:43.808467 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:43.808692 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:43.808808 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:43.811693 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:43.811697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:43.811699 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:43.811722 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:43.813632 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:43.814161 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:43.814164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:43.814166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:43.814305 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:43.814686 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:43.814689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:43.814691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:43.814889 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:43.815013 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:43.815557 (+ 544us) server_negotiation.cc:299] Negotiation successful
0808 18:27:43.815687 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 18:27:43.816628 13736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:43.802562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:46189 (local address 127.7.164.66:43507)
0808 18:27:43.803105 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:43.803163 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:43.804751 (+ 1588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:43.805287 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:43.805297 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:43.805823 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:43.806569 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:43.806593 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:43.807437 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:43.807441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:43.808321 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:43.808332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:43.808849 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:43.808853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:43.809056 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:43.809840 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:43.809864 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:43.811495 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:43.813764 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:43.813778 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:43.813791 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:43.814045 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:43.814412 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:43.814415 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:43.814417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:43.814573 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:43.815013 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:43.815021 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:43.815264 (+ 243us) client_negotiation.cc:769] Sending connection context
0808 18:27:43.815485 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 18:27:43.815819 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
I20260808 18:27:43.817821 13732 heartbeater.cc:347] Connected to a master server at 127.7.164.126:46189
I20260808 18:27:43.818102 13732 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:43.818639 13732 heartbeater.cc:511] Master 127.7.164.126:46189 requested a full tablet report, sending...
I20260808 18:27:43.819779 13251 ts_manager.cc:194] Registered new tserver with Master: 3be45c0ff21e46e0a8cd2fbfcc744381 (127.7.164.66:37035)
I20260808 18:27:43.820621 13251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:43507
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:27:43.833577 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:46189
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:43.949765 13740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:43.950080 13740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:43.950126 13740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:43.954149 13740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:43.954205 13740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:43.954319 13740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:27:43.959517 13740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:46189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:43.960567 13740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:43.961544 13740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:43.971428 13745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:43.971428 13748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:43.971557 13740 server_base.cc:1034] running on GCE node
W20260808 18:27:43.971441 13751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:43.972220 13740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:43.972815 13740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:43.974038 13740 hybrid_clock.cc:648] HybridClock initialized: now 1786213663974016 us; error 40 us; skew 500 ppm
Aug 08 18:27:43 dist-test-slave-jztv krb5kdc[13183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213663, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:43.976897 13740 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:27:43.977731 13740 webserver.cc:460] Webserver started at http://127.7.164.67:45247/ using document root <none> and password file <none>
I20260808 18:27:43.978319 13740 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:43.978389 13740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:43.978567 13740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:43.980521 13740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "04ba157b0def4f7fadf2dd4b13050fbe"
format_stamp: "Formatted at 2026-08-08 18:27:43 on dist-test-slave-jztv"
server_key: "8be018c86e6f73c29fc0e52d4135da34"
server_key_iv: "757d5c0a7d7d30c034f8602e993e5b4f"
server_key_version: "encryptionkey@0"
I20260808 18:27:43.981050 13740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "04ba157b0def4f7fadf2dd4b13050fbe"
format_stamp: "Formatted at 2026-08-08 18:27:43 on dist-test-slave-jztv"
server_key: "8be018c86e6f73c29fc0e52d4135da34"
server_key_iv: "757d5c0a7d7d30c034f8602e993e5b4f"
server_key_version: "encryptionkey@0"
I20260808 18:27:43.984764 13740 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:27:43.987102 13759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:43.988260 13740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:43.988354 13740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "04ba157b0def4f7fadf2dd4b13050fbe"
format_stamp: "Formatted at 2026-08-08 18:27:43 on dist-test-slave-jztv"
server_key: "8be018c86e6f73c29fc0e52d4135da34"
server_key_iv: "757d5c0a7d7d30c034f8602e993e5b4f"
server_key_version: "encryptionkey@0"
I20260808 18:27:43.988436 13740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:44.002576 13740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:44.005388 13740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:44.005539 13740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:44.006053 13740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:44.006898 13740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:44.006958 13740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:44.006997 13740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:44.007019 13740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:44.016187 13740 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:45253
I20260808 18:27:44.016218 13926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:45253 every 8 connection(s)
I20260808 18:27:44.017114 13740 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:27:44.021368 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 13740
I20260808 18:27:44.021521 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:27:44.021827 7825 external_mini_cluster.cc:1428] Setting key a1ca32e2444559e8b5eacf076b1ff01e
Aug 08 18:27:44 dist-test-slave-jztv krb5kdc[13183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213663, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:44.031281 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:44.018984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:40763 (local address 127.7.164.126:46189)
0808 18:27:44.019141 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:44.019145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:44.019978 (+ 833us) server_negotiation.cc:407] Connection header received
0808 18:27:44.021364 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:44.021369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:44.021434 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:44.021538 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:44.023149 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:44.023918 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:44.025062 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:44.025273 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:44.025354 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:44.027770 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:44.027774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:44.027776 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:44.027800 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:44.029321 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:44.029782 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:44.029785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:44.029787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:44.029862 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:44.030154 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:44.030156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:44.030157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:44.030404 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:44.030489 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:44.031005 (+ 516us) server_negotiation.cc:299] Negotiation successful
0808 18:27:44.031137 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:27:44.032047 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:44.019376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:46189 (local address 127.7.164.67:40763)
0808 18:27:44.019818 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:44.019860 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:44.021149 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:44.021794 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:44.021800 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:44.022324 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:44.023007 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:44.023023 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:44.024060 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:44.024063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:44.024950 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:44.024959 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:44.025402 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:44.025405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:44.025568 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:44.026122 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:44.026139 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:44.027614 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:44.029423 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:44.029427 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:44.029439 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:44.029696 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:44.029943 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:44.029945 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:44.029946 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:44.030076 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:44.030534 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:44.030542 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:44.030787 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 18:27:44.030947 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 18:27:44.031199 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260808 18:27:44.033361 13927 heartbeater.cc:347] Connected to a master server at 127.7.164.126:46189
I20260808 18:27:44.033627 13927 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:44.034283 13927 heartbeater.cc:511] Master 127.7.164.126:46189 requested a full tablet report, sending...
I20260808 18:27:44.035509 13251 ts_manager.cc:194] Registered new tserver with Master: 04ba157b0def4f7fadf2dd4b13050fbe (127.7.164.67:45253)
I20260808 18:27:44.036224 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:27:44.036221 13251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:40763
Aug 08 18:27:44 dist-test-slave-jztv krb5kdc[13183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:27:44 dist-test-slave-jztv krb5kdc[13183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.65@KRBTEST.COM
I20260808 18:27:44.065320 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:44.055210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49014 (local address 127.7.164.65:43437)
0808 18:27:44.055587 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:44.055602 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:44.055628 (+ 26us) server_negotiation.cc:407] Connection header received
0808 18:27:44.055680 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:44.055685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:44.055880 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:44.056006 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:44.056926 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:44.057720 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:44.058699 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:44.059086 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:44.059185 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:44.061731 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:44.061741 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:44.061754 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:44.061780 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:44.063553 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:44.063983 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:44.063988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:44.063990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:44.064073 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:44.064395 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:44.064398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:44.064400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:44.064691 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:44.064884 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:44.065027 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 18:27:44.065157 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":153,"threads_started":1}
Aug 08 18:27:44 dist-test-slave-jztv krb5kdc[13183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:44.077781 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:44.068254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52586 (local address 127.7.164.126:46189)
0808 18:27:44.068391 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:44.068397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:44.068471 (+ 74us) server_negotiation.cc:407] Connection header received
0808 18:27:44.068635 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:44.068639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:44.068693 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:44.068777 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:44.069361 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:44.069963 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:44.071146 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:44.071510 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:44.071627 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:44.074522 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:44.074525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:44.074527 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:44.074553 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:44.075983 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:44.076414 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:44.076417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:44.076418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:44.076519 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:44.076840 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:44.076843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:44.076844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:44.077111 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:44.077231 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:44.077454 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 18:27:44.077553 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 18:27:44.078295 13251 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52586
I20260808 18:27:44.078888 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 13344
I20260808 18:27:44.090375 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 13534
I20260808 18:27:44.103979 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 13740
I20260808 18:27:44.116688 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 13203
2026-08-08T18:27:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2370 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:44 dist-test-slave-jztv krb5kdc[13948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:44 dist-test-slave-jztv krb5kdc[13948](info): set up 2 sockets
Aug 08 18:27:44 dist-test-slave-jztv krb5kdc[13948](info): commencing operation
krb5kdc: starting...
W20260808 18:27:45.356884 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:27:45 dist-test-slave-jztv krb5kdc[13948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:27:45.490489 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:43173
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:43173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:45.602726 13964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:45.603068 13964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:45.603159 13964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:45.607504 13964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:45.607613 13964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:45.607654 13964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:45.607724 13964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:45.607764 13964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:45.613296 13964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:43173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:43173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:45.614519 13964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:45.615597 13964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:45.624351 13970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:45.624399 13973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:45.624389 13969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:45.624781 13964 server_base.cc:1034] running on GCE node
I20260808 18:27:45.625295 13964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:45.626451 13964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:45.627637 13964 hybrid_clock.cc:648] HybridClock initialized: now 1786213665627626 us; error 28 us; skew 500 ppm
Aug 08 18:27:45 dist-test-slave-jztv krb5kdc[13948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213665, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:45.630494 13964 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:27:45.631242 13964 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:45.631338 13964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:45.631587 13964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:45.633685 13964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "bfd4707549b14a9db18c73fb7e515158"
format_stamp: "Formatted at 2026-08-08 18:27:45 on dist-test-slave-jztv"
server_key: "5bb8bb1a4b092b1e6cba61755e0e9427"
server_key_iv: "84209c1d7f061b779ac7cde499e04509"
server_key_version: "encryptionkey@0"
I20260808 18:27:45.634351 13964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "bfd4707549b14a9db18c73fb7e515158"
format_stamp: "Formatted at 2026-08-08 18:27:45 on dist-test-slave-jztv"
server_key: "5bb8bb1a4b092b1e6cba61755e0e9427"
server_key_iv: "84209c1d7f061b779ac7cde499e04509"
server_key_version: "encryptionkey@0"
I20260808 18:27:45.638037 13964 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:27:45.640678 13980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:45.642041 13964 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:45.642190 13964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "bfd4707549b14a9db18c73fb7e515158"
format_stamp: "Formatted at 2026-08-08 18:27:45 on dist-test-slave-jztv"
server_key: "5bb8bb1a4b092b1e6cba61755e0e9427"
server_key_iv: "84209c1d7f061b779ac7cde499e04509"
server_key_version: "encryptionkey@0"
I20260808 18:27:45.642359 13964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:45.659582 13964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:45.662604 13964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:45.662819 13964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:45.670097 13964 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:43173
I20260808 18:27:45.670389 14060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:43173 every 8 connection(s)
I20260808 18:27:45.670876 13964 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:45.673604 14061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:45.678081 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 13964
I20260808 18:27:45.678228 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:45.678527 7825 external_mini_cluster.cc:1428] Setting key 719291306123013446904b5f7424be0d
I20260808 18:27:45.679692 14061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158: Bootstrap starting.
I20260808 18:27:45.682008 14061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:45.683146 14061 log.cc:824] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:45.685391 14061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158: No bootstrap required, opened a new log
Aug 08 18:27:45 dist-test-slave-jztv krb5kdc[13948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:45.689836 14061 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd4707549b14a9db18c73fb7e515158" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 43173 } }
I20260808 18:27:45.690119 14061 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:45.690182 14061 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfd4707549b14a9db18c73fb7e515158, State: Initialized, Role: FOLLOWER
I20260808 18:27:45.690737 14061 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd4707549b14a9db18c73fb7e515158" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 43173 } }
I20260808 18:27:45.690865 14061 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:45.690928 14061 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:45.691030 14061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:45.691917 14061 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd4707549b14a9db18c73fb7e515158" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 43173 } }
I20260808 18:27:45.692253 14061 leader_election.cc:302] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfd4707549b14a9db18c73fb7e515158; no voters:
I20260808 18:27:45.692704 14061 leader_election.cc:288] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:45.692800 14068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:45.693015 14068 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [term 1 LEADER]: Becoming Leader. State: Replica: bfd4707549b14a9db18c73fb7e515158, State: Running, Role: LEADER
I20260808 18:27:45.693435 14068 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd4707549b14a9db18c73fb7e515158" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 43173 } }
I20260808 18:27:45.694252 14061 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:45.694790 14074 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfd4707549b14a9db18c73fb7e515158. Latest consensus state: current_term: 1 leader_uuid: "bfd4707549b14a9db18c73fb7e515158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd4707549b14a9db18c73fb7e515158" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 43173 } } }
I20260808 18:27:45.694923 14074 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:45.695225 14070 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfd4707549b14a9db18c73fb7e515158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd4707549b14a9db18c73fb7e515158" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 43173 } } }
I20260808 18:27:45.695367 14070 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:45.695366 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:45.680042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46852 (local address 127.7.164.126:43173)
0808 18:27:45.680473 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:45.680489 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:45.680527 (+ 38us) server_negotiation.cc:407] Connection header received
0808 18:27:45.681186 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:45.681202 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:45.681542 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:45.681824 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:45.683003 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:45.684273 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:45.685571 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:45.686062 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:45.686186 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:45.688980 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:45.688985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:45.688998 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:45.689026 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:45.691223 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:45.691909 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:45.691915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:45.691921 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:45.692096 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:45.692426 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:45.692430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:45.692432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:45.693015 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:45.693855 (+ 840us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:45.694010 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 18:27:45.694340 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":139,"threads_started":1}
W20260808 18:27:45.699956 14093 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:27:45.700047 14093 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:27:45.700106 14076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:45.701010 14076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:45.706923 14076 catalog_manager.cc:1347] Generated new cluster ID: 6c7d6dd337fc4378b3c976adf2e0b381
I20260808 18:27:45.707056 14076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:45.716888 14076 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:45.717950 14076 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:45.733317 14076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158: Generated new TSK 0
I20260808 18:27:45.734151 14076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:27:45.773445 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:43173
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:45.889846 14101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:45.890118 14101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:45.890165 14101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:45.894316 14101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:45.894389 14101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:45.894450 14101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:27:45.900543 14101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:43173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:45.901615 14101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:45.902637 14101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:45.912473 14115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:45.912456 14111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:45.912456 14112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:45.912660 14101 server_base.cc:1034] running on GCE node
I20260808 18:27:45.913153 14101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:45.913833 14101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:45.915036 14101 hybrid_clock.cc:648] HybridClock initialized: now 1786213665915012 us; error 41 us; skew 500 ppm
Aug 08 18:27:45 dist-test-slave-jztv krb5kdc[13948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213665, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:45.917955 14101 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:27:45.918756 14101 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:45.918824 14101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:45.919019 14101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:45.921119 14101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "a0d7b714f8a94fd18fe086ea49363897"
format_stamp: "Formatted at 2026-08-08 18:27:45 on dist-test-slave-jztv"
server_key: "0d6279ae4dc4bfb9b72d4ee55783c4aa"
server_key_iv: "55a3cba0be63b3dafdf5b55a3be1d362"
server_key_version: "encryptionkey@0"
I20260808 18:27:45.921677 14101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "a0d7b714f8a94fd18fe086ea49363897"
format_stamp: "Formatted at 2026-08-08 18:27:45 on dist-test-slave-jztv"
server_key: "0d6279ae4dc4bfb9b72d4ee55783c4aa"
server_key_iv: "55a3cba0be63b3dafdf5b55a3be1d362"
server_key_version: "encryptionkey@0"
I20260808 18:27:45.925473 14101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:27:45.927946 14126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:45.929177 14101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:27:45.929278 14101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "a0d7b714f8a94fd18fe086ea49363897"
format_stamp: "Formatted at 2026-08-08 18:27:45 on dist-test-slave-jztv"
server_key: "0d6279ae4dc4bfb9b72d4ee55783c4aa"
server_key_iv: "55a3cba0be63b3dafdf5b55a3be1d362"
server_key_version: "encryptionkey@0"
I20260808 18:27:45.929368 14101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:45.940937 14101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:45.943769 14101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:45.943918 14101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:45.944566 14101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:45.945640 14101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:45.945708 14101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:45.945776 14101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:45.945807 14101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:45.954461 14101 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:41641
I20260808 18:27:45.954481 14306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:41641 every 8 connection(s)
I20260808 18:27:45.955057 14101 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:27:45.960835 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 14101
I20260808 18:27:45.960968 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:27:45.961261 7825 external_mini_cluster.cc:1428] Setting key 2748538467ee95939d0764cf7da9ee80
Aug 08 18:27:45 dist-test-slave-jztv krb5kdc[13948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213665, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:45.969007 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:45.956689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:59971 (local address 127.7.164.126:43173)
0808 18:27:45.956824 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:45.956827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:45.957999 (+ 1172us) server_negotiation.cc:407] Connection header received
0808 18:27:45.958978 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:45.958981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:45.959043 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:45.959125 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:45.960297 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:45.960871 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:45.962114 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:45.962470 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:45.962586 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:45.965127 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:45.965130 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:45.965132 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:45.965158 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:45.966683 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:45.967365 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:45.967367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:45.967369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:45.967446 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:45.967858 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:45.967861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:45.967862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:45.968038 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:45.968114 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:45.968778 (+ 664us) server_negotiation.cc:299] Negotiation successful
0808 18:27:45.968879 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":49}
I20260808 18:27:45.970389 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:45.957047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:43173 (local address 127.7.164.65:59971)
0808 18:27:45.957851 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:45.957896 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:45.958769 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:45.959291 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:45.959298 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:45.959674 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:45.960145 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:45.960159 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:45.961011 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:45.961015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:45.961981 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:45.961991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:45.962627 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:45.962632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:45.962809 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:45.963435 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:45.963453 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:45.964951 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:45.966806 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:45.966815 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:45.966833 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:45.967258 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:45.967539 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:45.967546 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:45.967551 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:45.967757 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:45.968148 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:45.968158 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:45.968512 (+ 354us) client_negotiation.cc:769] Sending connection context
0808 18:27:45.968754 (+ 242us) client_negotiation.cc:240] Negotiation successful
0808 18:27:45.969166 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":115,"threads_started":1}
I20260808 18:27:45.971712 14307 heartbeater.cc:347] Connected to a master server at 127.7.164.126:43173
I20260808 18:27:45.971952 14307 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:27:45.972441 14307 heartbeater.cc:511] Master 127.7.164.126:43173 requested a full tablet report, sending...
I20260808 18:27:45.973889 14005 ts_manager.cc:194] Registered new tserver with Master: a0d7b714f8a94fd18fe086ea49363897 (127.7.164.65:41641)
I20260808 18:27:45.975131 14005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:59971
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:27:45.992933 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:43173
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:46.108364 14314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:46.108671 14314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:46.108719 14314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:46.112676 14314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:46.112728 14314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:46.112785 14314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:27:46.117846 14314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:43173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:46.118867 14314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:46.119822 14314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:46.129066 14326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:46.129070 14325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:46.129127 14330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:46.129444 14314 server_base.cc:1034] running on GCE node
I20260808 18:27:46.129873 14314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:46.130559 14314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:46.131739 14314 hybrid_clock.cc:648] HybridClock initialized: now 1786213666131718 us; error 34 us; skew 500 ppm
Aug 08 18:27:46 dist-test-slave-jztv krb5kdc[13948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213666, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:46.134644 14314 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:27:46.135366 14314 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:46.135456 14314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:46.135686 14314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:46.137697 14314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "f885599a3c2e483eab6f88df50e46861"
format_stamp: "Formatted at 2026-08-08 18:27:46 on dist-test-slave-jztv"
server_key: "4925b3a5ec088343313ee38017d5ad0b"
server_key_iv: "20f52cfaf7d59f2587b5cff05f073025"
server_key_version: "encryptionkey@0"
I20260808 18:27:46.138334 14314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "f885599a3c2e483eab6f88df50e46861"
format_stamp: "Formatted at 2026-08-08 18:27:46 on dist-test-slave-jztv"
server_key: "4925b3a5ec088343313ee38017d5ad0b"
server_key_iv: "20f52cfaf7d59f2587b5cff05f073025"
server_key_version: "encryptionkey@0"
I20260808 18:27:46.142174 14314 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:27:46.144662 14337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:46.145926 14314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:46.146067 14314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "f885599a3c2e483eab6f88df50e46861"
format_stamp: "Formatted at 2026-08-08 18:27:46 on dist-test-slave-jztv"
server_key: "4925b3a5ec088343313ee38017d5ad0b"
server_key_iv: "20f52cfaf7d59f2587b5cff05f073025"
server_key_version: "encryptionkey@0"
I20260808 18:27:46.146178 14314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:46.168987 14314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:46.172271 14314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:46.172490 14314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:46.173118 14314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:46.174021 14314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:46.174100 14314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:46.174170 14314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:46.174216 14314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:46.183396 14314 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:45637
I20260808 18:27:46.183413 14504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:45637 every 8 connection(s)
I20260808 18:27:46.184052 14314 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:27:46.190550 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 14314
I20260808 18:27:46.190692 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:27:46.191020 7825 external_mini_cluster.cc:1428] Setting key 630f998fc622a9691b14c9aa3dff8721
Aug 08 18:27:46 dist-test-slave-jztv krb5kdc[13948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213666, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:46.199672 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.185867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:60821 (local address 127.7.164.126:43173)
0808 18:27:46.186045 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:46.186050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:46.187330 (+ 1280us) server_negotiation.cc:407] Connection header received
0808 18:27:46.188168 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:46.188171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:46.188219 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:46.188295 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:46.189467 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.189992 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.191628 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.191893 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.191995 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.195303 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:46.195310 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:46.195314 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:46.195352 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:46.197290 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:46.197905 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:46.197909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:46.197910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:46.197995 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:46.198440 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:46.198444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:46.198446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:46.198666 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:46.198777 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:46.199335 (+ 558us) server_negotiation.cc:299] Negotiation successful
0808 18:27:46.199482 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 18:27:46.200516 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.186317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:43173 (local address 127.7.164.66:60821)
0808 18:27:46.187173 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:46.187215 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:46.187987 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:46.188458 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:46.188464 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:46.188845 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:46.189338 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:46.189352 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.190111 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.190114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:46.191487 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:46.191497 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.192054 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.192057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:46.192201 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.192986 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:46.193008 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:46.195069 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:46.197444 (+ 2375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:46.197451 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:46.197470 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:46.197777 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:46.198096 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:46.198099 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:46.198101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:46.198331 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:46.198776 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:46.198785 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:46.199069 (+ 284us) client_negotiation.cc:769] Sending connection context
0808 18:27:46.199263 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 18:27:46.199637 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":122,"threads_started":1}
I20260808 18:27:46.201751 14505 heartbeater.cc:347] Connected to a master server at 127.7.164.126:43173
I20260808 18:27:46.202044 14505 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
I20260808 18:27:46.202674 14505 heartbeater.cc:511] Master 127.7.164.126:43173 requested a full tablet report, sending...
I20260808 18:27:46.203825 14005 ts_manager.cc:194] Registered new tserver with Master: f885599a3c2e483eab6f88df50e46861 (127.7.164.66:45637)
I20260808 18:27:46.204438 14005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:60821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:27:46.223261 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:43173
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:46.349529 14512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:46.349799 14512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:46.349886 14512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:46.354039 14512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:46.354106 14512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:46.354202 14512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:27:46.359601 14512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:43173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:46.360672 14512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:46.361707 14512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:46.371057 14519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:46.371045 14518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:46.371048 14522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:46.371383 14512 server_base.cc:1034] running on GCE node
I20260808 18:27:46.371799 14512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:46.372386 14512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:46.373590 14512 hybrid_clock.cc:648] HybridClock initialized: now 1786213666373571 us; error 30 us; skew 500 ppm
Aug 08 18:27:46 dist-test-slave-jztv krb5kdc[13948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213666, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:46.376539 14512 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:27:46.377334 14512 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:46.377429 14512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:46.377663 14512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:46.379760 14512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "786de5f9fd084213888d6797f36f8b4f"
format_stamp: "Formatted at 2026-08-08 18:27:46 on dist-test-slave-jztv"
server_key: "fcc2ddd22a6e9198f506487186ee0c29"
server_key_iv: "f71a2fd5ffcb1cc4fa8d185cb73888d6"
server_key_version: "encryptionkey@0"
I20260808 18:27:46.380348 14512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "786de5f9fd084213888d6797f36f8b4f"
format_stamp: "Formatted at 2026-08-08 18:27:46 on dist-test-slave-jztv"
server_key: "fcc2ddd22a6e9198f506487186ee0c29"
server_key_iv: "f71a2fd5ffcb1cc4fa8d185cb73888d6"
server_key_version: "encryptionkey@0"
I20260808 18:27:46.384310 14512 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:27:46.386806 14531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:46.388136 14512 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:46.388267 14512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "786de5f9fd084213888d6797f36f8b4f"
format_stamp: "Formatted at 2026-08-08 18:27:46 on dist-test-slave-jztv"
server_key: "fcc2ddd22a6e9198f506487186ee0c29"
server_key_iv: "f71a2fd5ffcb1cc4fa8d185cb73888d6"
server_key_version: "encryptionkey@0"
I20260808 18:27:46.388408 14512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:46.414593 14512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:46.417996 14512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:46.418253 14512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:46.418898 14512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:46.419782 14512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:46.419853 14512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:46.419921 14512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:46.419962 14512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:46.429190 14512 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:37817
I20260808 18:27:46.429210 14695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:37817 every 8 connection(s)
I20260808 18:27:46.429790 14512 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:27:46.431242 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 14512
I20260808 18:27:46.431381 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:27:46.431738 7825 external_mini_cluster.cc:1428] Setting key d6e8f7f80044bbb2df2c625bacc42603
Aug 08 18:27:46 dist-test-slave-jztv krb5kdc[13948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213666, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:46.443877 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.431691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:60025 (local address 127.7.164.126:43173)
0808 18:27:46.431864 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:46.431869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:46.433421 (+ 1552us) server_negotiation.cc:407] Connection header received
0808 18:27:46.434564 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:46.434567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:46.434614 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:46.434677 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:46.435902 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.436422 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.437592 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.437797 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.437868 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.440354 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:46.440357 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:46.440360 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:46.440385 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:46.441836 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:46.442373 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:46.442376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:46.442377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:46.442460 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:46.442808 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:46.442811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:46.442812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:46.443040 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:46.443140 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:46.443620 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 18:27:46.443738 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:27:46.444630 14700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.432162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:43173 (local address 127.7.164.67:60025)
0808 18:27:46.433251 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:46.433312 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:46.434360 (+ 1048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:46.434841 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:46.434851 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:46.435265 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:46.435776 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:46.435790 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.436529 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.436532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:46.437479 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:46.437488 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.437920 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.437922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:46.438070 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.438700 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:46.438716 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:46.440201 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:46.441945 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:46.441950 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:46.441962 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:46.442260 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:46.442540 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:46.442542 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:46.442543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:46.442677 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:46.443148 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:46.443153 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:46.443385 (+ 232us) client_negotiation.cc:769] Sending connection context
0808 18:27:46.443534 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 18:27:46.443780 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":118,"threads_started":1}
I20260808 18:27:46.445873 14697 heartbeater.cc:347] Connected to a master server at 127.7.164.126:43173
I20260808 18:27:46.446135 14697 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:46.446674 14697 heartbeater.cc:511] Master 127.7.164.126:43173 requested a full tablet report, sending...
I20260808 18:27:46.447676 14005 ts_manager.cc:194] Registered new tserver with Master: 786de5f9fd084213888d6797f36f8b4f (127.7.164.67:37817)
I20260808 18:27:46.448244 14005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:60025
I20260808 18:27:46.457758 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:27:46.467881 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.459747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46856 (local address 127.7.164.126:43173)
0808 18:27:46.459908 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:46.459915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:46.460092 (+ 177us) server_negotiation.cc:407] Connection header received
0808 18:27:46.460266 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:46.460269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:46.460312 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:46.460398 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:46.461199 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.461759 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.462721 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.462961 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.463098 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.464505 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:46.464509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:46.464513 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:46.464548 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:46.465949 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:46.466505 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:46.466510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:46.466513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:46.466604 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:46.466898 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:46.466904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:46.466906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:46.467161 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:46.467312 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:46.467574 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 18:27:46.467700 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:27:46.470695 14005 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:27:46.473886 14005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:27:46.488282 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.482315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:45637 (local address 127.0.0.1:49718)
0808 18:27:46.483014 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:46.483067 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:46.483206 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:46.484089 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:46.484095 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:46.484133 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:46.484325 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:46.484334 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.485330 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.485337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:46.486551 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:46.486562 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.487646 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.487650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:46.487782 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.487831 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 18:27:46.487956 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 18:27:46.488106 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":26,"spinlock_wait_cycles":7680,"thread_start_us":75,"threads_started":1}
I20260808 18:27:46.488286 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.482461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49718 (local address 127.7.164.66:45637)
0808 18:27:46.483523 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:46.483538 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:46.483574 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:27:46.483662 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:46.483668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:46.483888 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:46.484051 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:46.484457 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.485197 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.486693 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.487522 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.487631 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.487779 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:46.488002 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 18:27:46.488102 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":942,"thread_start_us":135,"threads_started":1}
I20260808 18:27:46.488636 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.482449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:37817 (local address 127.0.0.1:38458)
0808 18:27:46.483307 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:46.483323 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:46.483410 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:46.484100 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:46.484102 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:46.484134 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:46.484359 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:46.484371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.485940 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.485944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:46.487072 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:46.487083 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.488199 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.488203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:46.488301 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.488337 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 18:27:46.488398 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 18:27:46.488495 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":47,"spinlock_wait_cycles":8064,"thread_start_us":95,"threads_started":1}
I20260808 18:27:46.488744 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.482746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:41641 (local address 127.0.0.1:40142)
0808 18:27:46.483578 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:46.483594 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:46.483666 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:46.484420 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:46.484426 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:46.484449 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:46.484555 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:46.484566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.486178 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.486182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:46.487333 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:46.487344 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.488400 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.488404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:46.488509 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.488540 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 18:27:46.488591 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 18:27:46.488650 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":98,"threads_started":1}
I20260808 18:27:46.489225 14723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.483126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38458 (local address 127.7.164.67:37817)
0808 18:27:46.483611 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:46.483628 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:46.483652 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:27:46.483727 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:46.483732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:46.483936 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:46.484265 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:46.484507 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.485793 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.487204 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.488056 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.488738 (+ 682us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.488894 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:46.488952 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 18:27:46.489056 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":159,"threads_started":1}
I20260808 18:27:46.490331 14725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.483215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40142 (local address 127.7.164.65:41641)
0808 18:27:46.483922 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:46.483928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:46.483955 (+ 27us) server_negotiation.cc:407] Connection header received
0808 18:27:46.484034 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:46.484050 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:46.484257 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:46.484715 (+ 458us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:46.484856 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.486010 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.487468 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.488234 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.488862 (+ 628us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.489027 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:46.489093 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 18:27:46.489194 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":107,"threads_started":1}
I20260808 18:27:46.492127 14407 tablet_service.cc:1467] Processing CreateTablet for tablet 7cf13388ea9649d7a3433462873e31dd (DEFAULT_TABLE table=test-table [id=c802bbd1e9634d4295e32122cdabfcf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:46.492128 14598 tablet_service.cc:1467] Processing CreateTablet for tablet 7cf13388ea9649d7a3433462873e31dd (DEFAULT_TABLE table=test-table [id=c802bbd1e9634d4295e32122cdabfcf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:46.492802 14201 tablet_service.cc:1467] Processing CreateTablet for tablet 7cf13388ea9649d7a3433462873e31dd (DEFAULT_TABLE table=test-table [id=c802bbd1e9634d4295e32122cdabfcf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:46.493287 14598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cf13388ea9649d7a3433462873e31dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:46.493283 14407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cf13388ea9649d7a3433462873e31dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:46.493781 14201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cf13388ea9649d7a3433462873e31dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:46.500569 14726 tablet_bootstrap.cc:492] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897: Bootstrap starting.
I20260808 18:27:46.501938 14729 tablet_bootstrap.cc:492] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f: Bootstrap starting.
I20260808 18:27:46.502535 14728 tablet_bootstrap.cc:492] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861: Bootstrap starting.
I20260808 18:27:46.503052 14726 tablet_bootstrap.cc:654] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:46.503808 14726 log.cc:824] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:46.504071 14729 tablet_bootstrap.cc:654] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:46.504609 14728 tablet_bootstrap.cc:654] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:46.505321 14729 log.cc:824] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:46.505769 14726 tablet_bootstrap.cc:492] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897: No bootstrap required, opened a new log
I20260808 18:27:46.505774 14728 log.cc:824] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:46.506017 14726 ts_tablet_manager.cc:1397] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:27:46.507445 14729 tablet_bootstrap.cc:492] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f: No bootstrap required, opened a new log
I20260808 18:27:46.507648 14729 ts_tablet_manager.cc:1397] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:27:46.507786 14728 tablet_bootstrap.cc:492] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861: No bootstrap required, opened a new log
I20260808 18:27:46.507964 14728 ts_tablet_manager.cc:1397] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:27:46.510908 14728 raft_consensus.cc:348] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f885599a3c2e483eab6f88df50e46861" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45637 } } peers { permanent_uuid: "786de5f9fd084213888d6797f36f8b4f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37817 } } peers { permanent_uuid: "a0d7b714f8a94fd18fe086ea49363897" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41641 } }
I20260808 18:27:46.511162 14728 raft_consensus.cc:374] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:46.511240 14728 raft_consensus.cc:729] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f885599a3c2e483eab6f88df50e46861, State: Initialized, Role: FOLLOWER
I20260808 18:27:46.511133 14726 raft_consensus.cc:348] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f885599a3c2e483eab6f88df50e46861" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45637 } } peers { permanent_uuid: "786de5f9fd084213888d6797f36f8b4f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37817 } } peers { permanent_uuid: "a0d7b714f8a94fd18fe086ea49363897" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41641 } }
I20260808 18:27:46.511412 14726 raft_consensus.cc:374] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:46.511476 14726 raft_consensus.cc:729] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0d7b714f8a94fd18fe086ea49363897, State: Initialized, Role: FOLLOWER
I20260808 18:27:46.511770 14728 consensus_queue.cc:260] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f885599a3c2e483eab6f88df50e46861" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45637 } } peers { permanent_uuid: "786de5f9fd084213888d6797f36f8b4f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37817 } } peers { permanent_uuid: "a0d7b714f8a94fd18fe086ea49363897" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41641 } }
I20260808 18:27:46.511941 14726 consensus_queue.cc:260] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f885599a3c2e483eab6f88df50e46861" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45637 } } peers { permanent_uuid: "786de5f9fd084213888d6797f36f8b4f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37817 } } peers { permanent_uuid: "a0d7b714f8a94fd18fe086ea49363897" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41641 } }
I20260808 18:27:46.511907 14729 raft_consensus.cc:348] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f885599a3c2e483eab6f88df50e46861" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45637 } } peers { permanent_uuid: "786de5f9fd084213888d6797f36f8b4f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37817 } } peers { permanent_uuid: "a0d7b714f8a94fd18fe086ea49363897" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41641 } }
I20260808 18:27:46.512188 14729 raft_consensus.cc:374] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:46.512267 14729 raft_consensus.cc:729] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 786de5f9fd084213888d6797f36f8b4f, State: Initialized, Role: FOLLOWER
I20260808 18:27:46.512698 14307 heartbeater.cc:503] Master 127.7.164.126:43173 was elected leader, sending a full tablet report...
I20260808 18:27:46.512790 14729 consensus_queue.cc:260] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f885599a3c2e483eab6f88df50e46861" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45637 } } peers { permanent_uuid: "786de5f9fd084213888d6797f36f8b4f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37817 } } peers { permanent_uuid: "a0d7b714f8a94fd18fe086ea49363897" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41641 } }
I20260808 18:27:46.512858 14728 ts_tablet_manager.cc:1428] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:27:46.513232 14505 heartbeater.cc:503] Master 127.7.164.126:43173 was elected leader, sending a full tablet report...
I20260808 18:27:46.513765 14729 ts_tablet_manager.cc:1428] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260808 18:27:46.514122 14726 ts_tablet_manager.cc:1428] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260808 18:27:46.514142 14697 heartbeater.cc:503] Master 127.7.164.126:43173 was elected leader, sending a full tablet report...
I20260808 18:27:46.549882 14735 raft_consensus.cc:482] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:27:46.550143 14735 raft_consensus.cc:504] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f885599a3c2e483eab6f88df50e46861" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45637 } } peers { permanent_uuid: "786de5f9fd084213888d6797f36f8b4f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37817 } } peers { permanent_uuid: "a0d7b714f8a94fd18fe086ea49363897" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41641 } }
I20260808 18:27:46.551644 14735 leader_election.cc:288] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 786de5f9fd084213888d6797f36f8b4f (127.7.164.67:37817), a0d7b714f8a94fd18fe086ea49363897 (127.7.164.65:41641)
I20260808 18:27:46.555476 14723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.552060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:59155 (local address 127.7.164.67:37817)
0808 18:27:46.552193 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:46.552199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:46.552213 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:27:46.552386 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:46.552389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:46.552440 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:46.552641 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:46.552937 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.553435 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.554520 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.554922 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.555047 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.555104 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:46.555279 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 18:27:46.555351 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:27:46.555476 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.551864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:37817 (local address 127.7.164.66:59155)
0808 18:27:46.552124 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:46.552142 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:46.552241 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:46.552656 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:46.552660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:46.552679 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:46.552803 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:46.552809 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.553550 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.553554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:46.554387 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:46.554396 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.555047 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.555050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:46.555161 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.555178 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 18:27:46.555261 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 18:27:46.555333 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":1}
I20260808 18:27:46.555655 14739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.551962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:41641 (local address 127.7.164.66:34189)
0808 18:27:46.552427 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:46.552438 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:46.552552 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:46.552813 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:46.552815 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:46.552832 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:46.552955 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:46.552960 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.553688 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.553692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:46.554550 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:46.554560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.555221 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.555225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:46.555333 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.555351 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 18:27:46.555436 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 18:27:46.555511 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":154,"threads_started":1}
I20260808 18:27:46.555642 14725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.552093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:34189 (local address 127.7.164.65:41641)
0808 18:27:46.552236 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:46.552240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:46.552522 (+ 282us) server_negotiation.cc:407] Connection header received
0808 18:27:46.552666 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:46.552669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:46.552717 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:46.552800 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:46.553073 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.553570 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.554672 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.555097 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.555227 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.555281 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:46.555454 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 18:27:46.555531 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 18:27:46.556298 14234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cf13388ea9649d7a3433462873e31dd" candidate_uuid: "f885599a3c2e483eab6f88df50e46861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0d7b714f8a94fd18fe086ea49363897" is_pre_election: true
I20260808 18:27:46.556298 14629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cf13388ea9649d7a3433462873e31dd" candidate_uuid: "f885599a3c2e483eab6f88df50e46861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "786de5f9fd084213888d6797f36f8b4f" is_pre_election: true
I20260808 18:27:46.556658 14629 raft_consensus.cc:2454] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f885599a3c2e483eab6f88df50e46861 in term 0.
I20260808 18:27:46.556658 14234 raft_consensus.cc:2454] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f885599a3c2e483eab6f88df50e46861 in term 0.
I20260808 18:27:46.557222 14341 leader_election.cc:302] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 786de5f9fd084213888d6797f36f8b4f, a0d7b714f8a94fd18fe086ea49363897, f885599a3c2e483eab6f88df50e46861; no voters:
I20260808 18:27:46.557550 14735 raft_consensus.cc:2781] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:27:46.557657 14735 raft_consensus.cc:482] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:27:46.557736 14735 raft_consensus.cc:3037] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:46.559031 14735 raft_consensus.cc:504] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f885599a3c2e483eab6f88df50e46861" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45637 } } peers { permanent_uuid: "786de5f9fd084213888d6797f36f8b4f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37817 } } peers { permanent_uuid: "a0d7b714f8a94fd18fe086ea49363897" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41641 } }
I20260808 18:27:46.559484 14735 leader_election.cc:288] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [CANDIDATE]: Term 1 election: Requested vote from peers 786de5f9fd084213888d6797f36f8b4f (127.7.164.67:37817), a0d7b714f8a94fd18fe086ea49363897 (127.7.164.65:41641)
I20260808 18:27:46.559906 14629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cf13388ea9649d7a3433462873e31dd" candidate_uuid: "f885599a3c2e483eab6f88df50e46861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "786de5f9fd084213888d6797f36f8b4f"
I20260808 18:27:46.559902 14234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cf13388ea9649d7a3433462873e31dd" candidate_uuid: "f885599a3c2e483eab6f88df50e46861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0d7b714f8a94fd18fe086ea49363897"
I20260808 18:27:46.560043 14629 raft_consensus.cc:3037] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:46.560043 14234 raft_consensus.cc:3037] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:46.561398 14629 raft_consensus.cc:2454] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f885599a3c2e483eab6f88df50e46861 in term 1.
I20260808 18:27:46.561406 14234 raft_consensus.cc:2454] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f885599a3c2e483eab6f88df50e46861 in term 1.
I20260808 18:27:46.561769 14339 leader_election.cc:302] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 786de5f9fd084213888d6797f36f8b4f, a0d7b714f8a94fd18fe086ea49363897, f885599a3c2e483eab6f88df50e46861; no voters:
I20260808 18:27:46.561939 14735 raft_consensus.cc:2781] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:46.562194 14735 raft_consensus.cc:686] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [term 1 LEADER]: Becoming Leader. State: Replica: f885599a3c2e483eab6f88df50e46861, State: Running, Role: LEADER
I20260808 18:27:46.562606 14735 consensus_queue.cc:237] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f885599a3c2e483eab6f88df50e46861" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45637 } } peers { permanent_uuid: "786de5f9fd084213888d6797f36f8b4f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37817 } } peers { permanent_uuid: "a0d7b714f8a94fd18fe086ea49363897" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41641 } }
I20260808 18:27:46.565723 14005 catalog_manager.cc:5495] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 reported cstate change: term changed from 0 to 1, leader changed from <none> to f885599a3c2e483eab6f88df50e46861 (127.7.164.66). New cstate: current_term: 1 leader_uuid: "f885599a3c2e483eab6f88df50e46861" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f885599a3c2e483eab6f88df50e46861" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786de5f9fd084213888d6797f36f8b4f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0d7b714f8a94fd18fe086ea49363897" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41641 } health_report { overall_health: UNKNOWN } } }
I20260808 18:27:46.590659 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.587058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49720 (local address 127.7.164.66:45637)
0808 18:27:46.587193 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:46.587198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:46.587213 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:27:46.587350 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:46.587352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:46.587408 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:46.587518 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:46.587785 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.588295 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.589139 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:46.589360 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:46.589466 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:46.589791 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:46.590244 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:46.590379 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:46.590421 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 18:27:46.590481 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 18:27:46.596904 14629 raft_consensus.cc:1261] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f [term 1 FOLLOWER]: Refusing update from remote peer f885599a3c2e483eab6f88df50e46861: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:46.596904 14234 raft_consensus.cc:1261] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897 [term 1 FOLLOWER]: Refusing update from remote peer f885599a3c2e483eab6f88df50e46861: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:46.597743 14741 consensus_queue.cc:1035] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786de5f9fd084213888d6797f36f8b4f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:46.597939 14735 consensus_queue.cc:1035] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0d7b714f8a94fd18fe086ea49363897" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:46.605329 14744 mvcc.cc:205] Tried to move back new op lower bound from 7316331178373152768 to 7316331178243203072. Current Snapshot: MvccSnapshot[applied={T|T < 7316331178373152768}]
I20260808 18:27:46.614133 14003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:46.614356 14003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:46.616783 14003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bfd4707549b14a9db18c73fb7e515158: Sending DeleteTablet for 3 replicas of tablet 7cf13388ea9649d7a3433462873e31dd
I20260808 18:27:46.617534 14407 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cf13388ea9649d7a3433462873e31dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:46 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:49718
I20260808 18:27:46.617664 14598 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cf13388ea9649d7a3433462873e31dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:46 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:38458
I20260808 18:27:46.617877 14201 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cf13388ea9649d7a3433462873e31dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:46 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:40142
I20260808 18:27:46.618160 14759 tablet_replica.cc:330] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f: stopping tablet replica
I20260808 18:27:46.618371 14761 tablet_replica.cc:330] T 7cf13388ea9649d7a3433462873e31dd P a0d7b714f8a94fd18fe086ea49363897: stopping tablet replica
I20260808 18:27:46.618553 14759 raft_consensus.cc:2229] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:46.618698 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 14101
I20260808 18:27:46.618922 14759 raft_consensus.cc:2258] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:46.630434 14760 tablet_replica.cc:330] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861: stopping tablet replica
I20260808 18:27:46.630887 14760 raft_consensus.cc:2229] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:27:46.631922 14760 raft_consensus.cc:2258] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:46.632413 14759 ts_tablet_manager.cc:1905] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:46.633504 14760 ts_tablet_manager.cc:1905] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:46.635375 14759 ts_tablet_manager.cc:1918] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:27:46.635558 14759 log.cc:1196] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/7cf13388ea9649d7a3433462873e31dd
I20260808 18:27:46.635994 14759 ts_tablet_manager.cc:1939] T 7cf13388ea9649d7a3433462873e31dd P 786de5f9fd084213888d6797f36f8b4f: Deleting consensus metadata
I20260808 18:27:46.637140 14760 ts_tablet_manager.cc:1918] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:27:46.637326 14760 log.cc:1196] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/7cf13388ea9649d7a3433462873e31dd
I20260808 18:27:46.637428 13982 catalog_manager.cc:4841] TS 786de5f9fd084213888d6797f36f8b4f (127.7.164.67:37817): tablet 7cf13388ea9649d7a3433462873e31dd (table test-table [id=c802bbd1e9634d4295e32122cdabfcf7]) successfully deleted
I20260808 18:27:46.637756 14760 ts_tablet_manager.cc:1939] T 7cf13388ea9649d7a3433462873e31dd P f885599a3c2e483eab6f88df50e46861: Deleting consensus metadata
W20260808 18:27:46.637971 13983 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:41641 (error 108)
I20260808 18:27:46.639034 13985 catalog_manager.cc:4841] TS f885599a3c2e483eab6f88df50e46861 (127.7.164.66:45637): tablet 7cf13388ea9649d7a3433462873e31dd (table test-table [id=c802bbd1e9634d4295e32122cdabfcf7]) successfully deleted
I20260808 18:27:46.639079 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.638772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:41641 (local address 127.0.0.1:40154)
0808 18:27:46.638920 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:46.638991 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:41641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 18:27:46.639279 13983 catalog_manager.cc:4584] TS a0d7b714f8a94fd18fe086ea49363897 (127.7.164.65:41641): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7cf13388ea9649d7a3433462873e31dd: Network error: Client connection negotiation failed: client connection to 127.7.164.65:41641: connect: Connection refused (error 111)
I20260808 18:27:46.639756 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 14314
I20260808 18:27:46.654052 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 14512
I20260808 18:27:46.665553 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 13964
2026-08-08T18:27:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2551 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T18:27:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:46Z Disabled control of system clock
I20260808 18:27:46.702553 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:41571
--webserver_interface=127.7.164.126
--webserver_port=0
--builtin_ntp_servers=127.7.164.84:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:41571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 18:27:46.815300 14766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:46.815573 14766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:46.815611 14766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:46.819774 14766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:46.819839 14766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:46.819854 14766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:46.819870 14766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:46.819885 14766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:46.824754 14766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:41571
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:41571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:46.825888 14766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:46.827334 14766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:46.836264 14773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:46.836282 14776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:46.836282 14772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:46.836345 14766 server_base.cc:1034] running on GCE node
I20260808 18:27:46.837091 14766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:46.838368 14766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:46.839574 14766 hybrid_clock.cc:648] HybridClock initialized: now 1786213666839541 us; error 44 us; skew 500 ppm
I20260808 18:27:46.844458 14766 webserver.cc:460] Webserver started at http://127.7.164.126:38621/ using document root <none> and password file <none>
I20260808 18:27:46.845075 14766 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:46.845170 14766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:46.845419 14766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:46.847599 14766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "eb0849da9c1a449997b3ea024633ed66"
format_stamp: "Formatted at 2026-08-08 18:27:46 on dist-test-slave-jztv"
server_key: "1da472e6fb5193dc55a54cf4a7e429b7"
server_key_iv: "e28d2b288a4ef7bd336e4ab9e6a2e8a7"
server_key_version: "encryptionkey@0"
I20260808 18:27:46.848237 14766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "eb0849da9c1a449997b3ea024633ed66"
format_stamp: "Formatted at 2026-08-08 18:27:46 on dist-test-slave-jztv"
server_key: "1da472e6fb5193dc55a54cf4a7e429b7"
server_key_iv: "e28d2b288a4ef7bd336e4ab9e6a2e8a7"
server_key_version: "encryptionkey@0"
I20260808 18:27:46.851939 14766 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:27:46.854463 14783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:46.855662 14766 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:46.855801 14766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "eb0849da9c1a449997b3ea024633ed66"
format_stamp: "Formatted at 2026-08-08 18:27:46 on dist-test-slave-jztv"
server_key: "1da472e6fb5193dc55a54cf4a7e429b7"
server_key_iv: "e28d2b288a4ef7bd336e4ab9e6a2e8a7"
server_key_version: "encryptionkey@0"
I20260808 18:27:46.855921 14766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:46.860813 14766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:46.861411 14766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:46.861588 14766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:46.868532 14766 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:41571
I20260808 18:27:46.868541 14860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:41571 every 8 connection(s)
I20260808 18:27:46.869454 14766 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:46.872726 14861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:46.878139 14861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66: Bootstrap starting.
I20260808 18:27:46.878507 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 14766
I20260808 18:27:46.878643 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:46.878942 7825 external_mini_cluster.cc:1428] Setting key 378e58ccd17bb9f67f8f66de8dce039d
I20260808 18:27:46.881029 14861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:46.881753 14861 log.cc:824] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:46.884075 14861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66: No bootstrap required, opened a new log
I20260808 18:27:46.884773 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:46.880346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49976 (local address 127.7.164.126:41571)
0808 18:27:46.881059 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:46.881079 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:46.881115 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:27:46.881803 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:46.881820 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:46.882284 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:46.882654 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:46.883467 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:46.883472 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:46.883484 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:46.883494 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:46.883551 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:46.883567 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:46.883656 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:46.883791 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 18:27:46.884052 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":142,"threads_started":1}
I20260808 18:27:46.888260 14861 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0849da9c1a449997b3ea024633ed66" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41571 } }
I20260808 18:27:46.888526 14861 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:46.888602 14861 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0849da9c1a449997b3ea024633ed66, State: Initialized, Role: FOLLOWER
I20260808 18:27:46.889055 14861 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0849da9c1a449997b3ea024633ed66" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41571 } }
I20260808 18:27:46.889202 14861 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:46.889282 14861 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:46.889389 14861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:46.890440 14861 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0849da9c1a449997b3ea024633ed66" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41571 } }
I20260808 18:27:46.890784 14861 leader_election.cc:302] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb0849da9c1a449997b3ea024633ed66; no voters:
I20260808 18:27:46.891094 14861 leader_election.cc:288] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:46.891232 14869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:46.891461 14869 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [term 1 LEADER]: Becoming Leader. State: Replica: eb0849da9c1a449997b3ea024633ed66, State: Running, Role: LEADER
I20260808 18:27:46.891841 14869 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0849da9c1a449997b3ea024633ed66" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41571 } }
I20260808 18:27:46.892304 14861 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:46.893558 14872 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb0849da9c1a449997b3ea024633ed66. Latest consensus state: current_term: 1 leader_uuid: "eb0849da9c1a449997b3ea024633ed66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0849da9c1a449997b3ea024633ed66" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41571 } } }
I20260808 18:27:46.893695 14872 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:46.893976 14870 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb0849da9c1a449997b3ea024633ed66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0849da9c1a449997b3ea024633ed66" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41571 } } }
I20260808 18:27:46.894074 14870 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:46.894100 14882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:46.897430 14882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:46.903134 14882 catalog_manager.cc:1347] Generated new cluster ID: 10ba29426f0b48c2a063ec9e48426c45
I20260808 18:27:46.903198 14882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:46.923877 14882 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:46.924259 14882 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:46.935053 14882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66: Generated new TSK 0
I20260808 18:27:46.935771 14882 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:27:46.942401 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41571
--builtin_ntp_servers=127.7.164.84:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 18:27:47.058522 14897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:47.058809 14897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:47.058880 14897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:47.063043 14897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:47.063102 14897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:47.063184 14897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:27:47.068030 14897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:47.069129 14897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:47.070595 14897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:47.080264 14903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:47.080265 14908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:47.080276 14905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:47.080688 14897 server_base.cc:1034] running on GCE node
I20260808 18:27:47.081110 14897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:47.081940 14897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:47.083112 14897 hybrid_clock.cc:648] HybridClock initialized: now 1786213667083085 us; error 36 us; skew 500 ppm
I20260808 18:27:47.085191 14897 webserver.cc:460] Webserver started at http://127.7.164.65:42795/ using document root <none> and password file <none>
I20260808 18:27:47.085738 14897 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:47.085801 14897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:47.085985 14897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:47.088114 14897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "e71036b002e54152bf5e6ff0f5095cbd"
format_stamp: "Formatted at 2026-08-08 18:27:47 on dist-test-slave-jztv"
server_key: "a98b3b249f6225081a9fd9661bf64de8"
server_key_iv: "45b195257c394afb7630abee093836c7"
server_key_version: "encryptionkey@0"
I20260808 18:27:47.088635 14897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "e71036b002e54152bf5e6ff0f5095cbd"
format_stamp: "Formatted at 2026-08-08 18:27:47 on dist-test-slave-jztv"
server_key: "a98b3b249f6225081a9fd9661bf64de8"
server_key_iv: "45b195257c394afb7630abee093836c7"
server_key_version: "encryptionkey@0"
I20260808 18:27:47.092433 14897 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:27:47.095104 14915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:47.096344 14897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:47.096478 14897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "e71036b002e54152bf5e6ff0f5095cbd"
format_stamp: "Formatted at 2026-08-08 18:27:47 on dist-test-slave-jztv"
server_key: "a98b3b249f6225081a9fd9661bf64de8"
server_key_iv: "45b195257c394afb7630abee093836c7"
server_key_version: "encryptionkey@0"
I20260808 18:27:47.096592 14897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:47.101222 14897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:47.101814 14897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:47.102000 14897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:47.102627 14897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:47.103535 14897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:47.103602 14897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:47.103670 14897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:47.103711 14897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:47.112397 14897 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:40943
I20260808 18:27:47.112430 15097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:40943 every 8 connection(s)
I20260808 18:27:47.113456 14897 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:27:47.119750 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.115294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:43763 (local address 127.7.164.126:41571)
0808 18:27:47.115440 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:47.115445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:47.116520 (+ 1075us) server_negotiation.cc:407] Connection header received
0808 18:27:47.117957 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:47.117961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:47.118009 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:47.118110 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:47.119258 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:47.119262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:47.119265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:47.119281 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:47.119299 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:47.119307 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:47.119413 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:47.119570 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 18:27:47.119614 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:27:47.119753 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 14897
I20260808 18:27:47.119894 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:27:47.120186 7825 external_mini_cluster.cc:1428] Setting key 83a1110eb5480f2230b5f34c31dc67c2
I20260808 18:27:47.120630 15101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.115719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41571 (local address 127.7.164.65:43763)
0808 18:27:47.116368 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:47.116409 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:47.117740 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:47.118321 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:47.118328 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:47.118413 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:47.119009 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:27:47.119010 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:47.119050 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:27:47.119114 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:47.119398 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:47.119407 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:47.119413 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:27:47.119613 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 18:27:47.119843 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":122,"threads_started":1}
I20260808 18:27:47.121833 15099 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41571
I20260808 18:27:47.122072 15099 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:47.122255 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41571
--builtin_ntp_servers=127.7.164.84:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 18:27:47.122557 15099 heartbeater.cc:511] Master 127.7.164.126:41571 requested a full tablet report, sending...
I20260808 18:27:47.123984 14802 ts_manager.cc:194] Registered new tserver with Master: e71036b002e54152bf5e6ff0f5095cbd (127.7.164.65:40943)
W20260808 18:27:47.235543 15102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:47.235839 15102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:47.235906 15102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:47.239931 15102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:47.240010 15102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:47.240101 15102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:27:47.244967 15102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:47.246083 15102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:47.247578 15102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:47.257321 15109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:47.257321 15110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:47.257364 15102 server_base.cc:1034] running on GCE node
W20260808 18:27:47.257349 15112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:47.258008 15102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:47.258736 15102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:47.259919 15102 hybrid_clock.cc:648] HybridClock initialized: now 1786213667259889 us; error 43 us; skew 500 ppm
I20260808 18:27:47.261821 15102 webserver.cc:460] Webserver started at http://127.7.164.66:39475/ using document root <none> and password file <none>
I20260808 18:27:47.262465 15102 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:47.262558 15102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:47.262789 15102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:47.264822 15102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "4b8ded84feda415bb23bd5a69fc21e4e"
format_stamp: "Formatted at 2026-08-08 18:27:47 on dist-test-slave-jztv"
server_key: "af2bd46f48a79441163e05315aaff11e"
server_key_iv: "f5d972cbcace8afa46f4c7f5e4bf83af"
server_key_version: "encryptionkey@0"
I20260808 18:27:47.265393 15102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "4b8ded84feda415bb23bd5a69fc21e4e"
format_stamp: "Formatted at 2026-08-08 18:27:47 on dist-test-slave-jztv"
server_key: "af2bd46f48a79441163e05315aaff11e"
server_key_iv: "f5d972cbcace8afa46f4c7f5e4bf83af"
server_key_version: "encryptionkey@0"
I20260808 18:27:47.269074 15102 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 18:27:47.271545 15125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:47.272699 15102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:27:47.272837 15102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "4b8ded84feda415bb23bd5a69fc21e4e"
format_stamp: "Formatted at 2026-08-08 18:27:47 on dist-test-slave-jztv"
server_key: "af2bd46f48a79441163e05315aaff11e"
server_key_iv: "f5d972cbcace8afa46f4c7f5e4bf83af"
server_key_version: "encryptionkey@0"
I20260808 18:27:47.272956 15102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:47.277462 15102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:47.278038 15102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:47.278206 15102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:47.278786 15102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:47.279747 15102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:47.279814 15102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:47.279884 15102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:47.279930 15102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:47.288592 15102 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:40303
I20260808 18:27:47.288628 15295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:40303 every 8 connection(s)
I20260808 18:27:47.289512 15102 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:27:47.295729 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.291348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:47225 (local address 127.7.164.126:41571)
0808 18:27:47.291499 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:47.291502 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:47.292601 (+ 1099us) server_negotiation.cc:407] Connection header received
0808 18:27:47.293984 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:47.293988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:47.294036 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:47.294133 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:47.295249 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:47.295253 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:47.295256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:47.295268 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:47.295288 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:47.295301 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:47.295393 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:47.295544 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 18:27:47.295590 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:27:47.296568 15298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.291736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41571 (local address 127.7.164.66:47225)
0808 18:27:47.292459 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:47.292500 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:47.293760 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:47.294350 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:47.294357 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:47.294446 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:47.295000 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:27:47.295002 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:47.295039 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:27:47.295104 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:47.295386 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:47.295396 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:47.295404 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 18:27:47.295576 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 18:27:47.295821 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":131,"threads_started":1}
I20260808 18:27:47.297688 15296 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41571
I20260808 18:27:47.297890 15296 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:47.298321 15296 heartbeater.cc:511] Master 127.7.164.126:41571 requested a full tablet report, sending...
I20260808 18:27:47.299273 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 15102
I20260808 18:27:47.299409 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:27:47.299407 14802 ts_manager.cc:194] Registered new tserver with Master: 4b8ded84feda415bb23bd5a69fc21e4e (127.7.164.66:40303)
I20260808 18:27:47.299723 7825 external_mini_cluster.cc:1428] Setting key 8501fe45628dbe6b3c142f1b7085db34
I20260808 18:27:47.301723 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41571
--builtin_ntp_servers=127.7.164.84:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 18:27:47.415805 15299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:47.416061 15299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:47.416101 15299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:47.420168 15299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:47.420228 15299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:47.420312 15299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:27:47.425056 15299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:47.426090 15299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:47.427464 15299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:47.437207 15309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:47.437214 15305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:47.437404 15299 server_base.cc:1034] running on GCE node
W20260808 18:27:47.437552 15304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:47.438045 15299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:47.438704 15299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:47.439905 15299 hybrid_clock.cc:648] HybridClock initialized: now 1786213667439887 us; error 28 us; skew 500 ppm
I20260808 18:27:47.441704 15299 webserver.cc:460] Webserver started at http://127.7.164.67:38895/ using document root <none> and password file <none>
I20260808 18:27:47.442349 15299 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:47.442438 15299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:47.442659 15299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:47.444730 15299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "288a5face5c14f1aae93eff72286dfbc"
format_stamp: "Formatted at 2026-08-08 18:27:47 on dist-test-slave-jztv"
server_key: "67857274da2525f146c6bb216311a259"
server_key_iv: "87c8416cd5f4ccc66e7a15e6c4d7df71"
server_key_version: "encryptionkey@0"
I20260808 18:27:47.445322 15299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "288a5face5c14f1aae93eff72286dfbc"
format_stamp: "Formatted at 2026-08-08 18:27:47 on dist-test-slave-jztv"
server_key: "67857274da2525f146c6bb216311a259"
server_key_iv: "87c8416cd5f4ccc66e7a15e6c4d7df71"
server_key_version: "encryptionkey@0"
I20260808 18:27:47.449070 15299 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 18:27:47.451584 15316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:47.453001 15299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 18:27:47.453137 15299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "288a5face5c14f1aae93eff72286dfbc"
format_stamp: "Formatted at 2026-08-08 18:27:47 on dist-test-slave-jztv"
server_key: "67857274da2525f146c6bb216311a259"
server_key_iv: "87c8416cd5f4ccc66e7a15e6c4d7df71"
server_key_version: "encryptionkey@0"
I20260808 18:27:47.453354 15299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:47.457877 15299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:47.458554 15299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:47.458724 15299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:47.459301 15299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:47.460199 15299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:47.460247 15299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:47.460309 15299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:47.460346 15299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:47.469571 15299 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:44681
I20260808 18:27:47.469590 15471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:44681 every 8 connection(s)
I20260808 18:27:47.470605 15299 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:27:47.476976 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.472457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:40759 (local address 127.7.164.126:41571)
0808 18:27:47.472615 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:47.472619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:47.473701 (+ 1082us) server_negotiation.cc:407] Connection header received
0808 18:27:47.475072 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:47.475074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:47.475113 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:47.475180 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:47.476397 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:47.476400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:47.476402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:47.476414 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:47.476437 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:47.476446 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:47.476535 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:47.476688 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 18:27:47.476787 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:27:47.477866 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.472834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41571 (local address 127.7.164.67:40759)
0808 18:27:47.473548 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:47.473590 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:47.474861 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:47.475351 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:47.475358 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:47.475464 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:47.476110 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:27:47.476111 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:47.476153 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:27:47.476225 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:47.476545 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:47.476555 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:47.476564 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 18:27:47.476711 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 18:27:47.476980 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":120,"threads_started":1}
I20260808 18:27:47.478822 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 15299
I20260808 18:27:47.478953 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:27:47.479174 15472 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41571
I20260808 18:27:47.479305 7825 external_mini_cluster.cc:1428] Setting key 4daf585ef00f0fdb6cec910b493b8873
I20260808 18:27:47.479457 15472 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:47.479974 15472 heartbeater.cc:511] Master 127.7.164.126:41571 requested a full tablet report, sending...
I20260808 18:27:47.481036 14805 ts_manager.cc:194] Registered new tserver with Master: 288a5face5c14f1aae93eff72286dfbc (127.7.164.67:44681)
I20260808 18:27:47.493611 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:27:47.497081 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.495688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49986 (local address 127.7.164.126:41571)
0808 18:27:47.495818 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:47.495821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:47.495954 (+ 133us) server_negotiation.cc:407] Connection header received
0808 18:27:47.496058 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:47.496060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:47.496094 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:47.496162 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:47.496797 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:47.496800 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:47.496802 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:47.496813 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:47.496830 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:47.496838 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:47.496906 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:47.496947 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 18:27:47.496979 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:27:47.499792 14805 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:27:47.503069 14805 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:27:47.515621 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.511966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57782 (local address 127.7.164.65:40943)
0808 18:27:47.512495 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:47.512507 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:47.512851 (+ 344us) server_negotiation.cc:407] Connection header received
0808 18:27:47.513130 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:47.513137 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:47.513331 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:47.513442 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:47.514935 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:47.514941 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:47.514954 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:47.514968 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:47.515012 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:47.515049 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:47.515160 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:47.515382 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 18:27:47.515454 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":132,"threads_started":1}
I20260808 18:27:47.515682 15493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.512688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:44681 (local address 127.0.0.1:52392)
0808 18:27:47.513156 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:47.513169 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:47.513237 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:47.513913 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:47.513920 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:47.513992 (+ 72us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:47.515001 (+ 1009us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:27:47.515003 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:47.515016 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:27:47.515035 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:47.515438 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:47.515441 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:47.515445 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:27:47.515503 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 18:27:47.515537 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":91,"threads_started":1}
I20260808 18:27:47.515547 15492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.511727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:40303 (local address 127.0.0.1:45518)
0808 18:27:47.512716 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:47.512770 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:47.512877 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:47.514099 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:47.514105 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:47.514124 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:47.514806 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:27:47.514809 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:47.514822 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:27:47.514856 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:47.515250 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:47.515252 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:47.515255 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 18:27:47.515336 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 18:27:47.515406 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"thread_start_us":77,"threads_started":1}
I20260808 18:27:47.515991 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.511727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:40943 (local address 127.0.0.1:57782)
0808 18:27:47.512714 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:47.512765 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:47.512919 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:47.514115 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:47.514118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:47.514134 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:47.514648 (+ 514us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:27:47.514652 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:47.514676 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:27:47.514751 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:47.515208 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:47.515217 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:47.515225 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 18:27:47.515800 (+ 575us) client_negotiation.cc:240] Negotiation successful
0808 18:27:47.515844 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"mutex_wait_us":610,"thread_start_us":105,"threads_started":1}
I20260808 18:27:47.516016 15495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.513134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52392 (local address 127.7.164.67:44681)
0808 18:27:47.513485 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:47.513496 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:47.513518 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:27:47.513568 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:47.513573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:47.513729 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:47.513869 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:47.515197 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:47.515203 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:47.515214 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:47.515226 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:47.515272 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:47.515305 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:47.515637 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:47.515687 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 18:27:47.515844 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":143,"threads_started":1}
I20260808 18:27:47.516644 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.511897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45518 (local address 127.7.164.66:40303)
0808 18:27:47.512213 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:47.512225 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:47.512998 (+ 773us) server_negotiation.cc:407] Connection header received
0808 18:27:47.513069 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:47.513075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:47.513289 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:47.513440 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:47.514989 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:47.514994 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:47.515003 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:47.515014 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:47.515086 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:47.515119 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:47.515214 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:47.516423 (+ 1209us) server_negotiation.cc:299] Negotiation successful
0808 18:27:47.516490 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":137,"threads_started":1}
I20260808 18:27:47.519074 15381 tablet_service.cc:1467] Processing CreateTablet for tablet 14e8e1f0a904417a97fee7e2d92e9b3d (DEFAULT_TABLE table=test-table [id=c14e817c0a9c48b5a1da0db39eeae159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:47.519073 14988 tablet_service.cc:1467] Processing CreateTablet for tablet 14e8e1f0a904417a97fee7e2d92e9b3d (DEFAULT_TABLE table=test-table [id=c14e817c0a9c48b5a1da0db39eeae159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:47.519172 15197 tablet_service.cc:1467] Processing CreateTablet for tablet 14e8e1f0a904417a97fee7e2d92e9b3d (DEFAULT_TABLE table=test-table [id=c14e817c0a9c48b5a1da0db39eeae159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:47.520184 15381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e8e1f0a904417a97fee7e2d92e9b3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:47.520215 15197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e8e1f0a904417a97fee7e2d92e9b3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:47.520215 14988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e8e1f0a904417a97fee7e2d92e9b3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:47.526530 15496 tablet_bootstrap.cc:492] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc: Bootstrap starting.
I20260808 18:27:47.527283 15497 tablet_bootstrap.cc:492] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd: Bootstrap starting.
I20260808 18:27:47.528412 15498 tablet_bootstrap.cc:492] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e: Bootstrap starting.
I20260808 18:27:47.529529 15496 tablet_bootstrap.cc:654] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:47.529626 15497 tablet_bootstrap.cc:654] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:47.530401 15496 log.cc:824] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:47.530516 15498 tablet_bootstrap.cc:654] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:47.530508 15497 log.cc:824] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:47.531265 15498 log.cc:824] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:47.532480 15497 tablet_bootstrap.cc:492] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd: No bootstrap required, opened a new log
I20260808 18:27:47.532506 15496 tablet_bootstrap.cc:492] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc: No bootstrap required, opened a new log
I20260808 18:27:47.532691 15497 ts_tablet_manager.cc:1397] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:27:47.532694 15496 ts_tablet_manager.cc:1397] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260808 18:27:47.533192 15498 tablet_bootstrap.cc:492] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e: No bootstrap required, opened a new log
I20260808 18:27:47.533354 15498 ts_tablet_manager.cc:1397] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:27:47.536473 15497 raft_consensus.cc:348] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8ded84feda415bb23bd5a69fc21e4e" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40303 } } peers { permanent_uuid: "e71036b002e54152bf5e6ff0f5095cbd" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 40943 } } peers { permanent_uuid: "288a5face5c14f1aae93eff72286dfbc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 44681 } }
I20260808 18:27:47.536732 15497 raft_consensus.cc:374] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:47.536796 15497 raft_consensus.cc:729] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e71036b002e54152bf5e6ff0f5095cbd, State: Initialized, Role: FOLLOWER
I20260808 18:27:47.537261 15497 consensus_queue.cc:260] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8ded84feda415bb23bd5a69fc21e4e" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40303 } } peers { permanent_uuid: "e71036b002e54152bf5e6ff0f5095cbd" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 40943 } } peers { permanent_uuid: "288a5face5c14f1aae93eff72286dfbc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 44681 } }
I20260808 18:27:47.537304 15496 raft_consensus.cc:348] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8ded84feda415bb23bd5a69fc21e4e" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40303 } } peers { permanent_uuid: "e71036b002e54152bf5e6ff0f5095cbd" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 40943 } } peers { permanent_uuid: "288a5face5c14f1aae93eff72286dfbc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 44681 } }
I20260808 18:27:47.537572 15496 raft_consensus.cc:374] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:47.537644 15496 raft_consensus.cc:729] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 288a5face5c14f1aae93eff72286dfbc, State: Initialized, Role: FOLLOWER
I20260808 18:27:47.537705 15498 raft_consensus.cc:348] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8ded84feda415bb23bd5a69fc21e4e" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40303 } } peers { permanent_uuid: "e71036b002e54152bf5e6ff0f5095cbd" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 40943 } } peers { permanent_uuid: "288a5face5c14f1aae93eff72286dfbc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 44681 } }
I20260808 18:27:47.537928 15498 raft_consensus.cc:374] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:47.537997 15498 raft_consensus.cc:729] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b8ded84feda415bb23bd5a69fc21e4e, State: Initialized, Role: FOLLOWER
I20260808 18:27:47.538077 15496 consensus_queue.cc:260] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8ded84feda415bb23bd5a69fc21e4e" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40303 } } peers { permanent_uuid: "e71036b002e54152bf5e6ff0f5095cbd" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 40943 } } peers { permanent_uuid: "288a5face5c14f1aae93eff72286dfbc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 44681 } }
I20260808 18:27:47.538389 15497 ts_tablet_manager.cc:1428] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 18:27:47.538415 15498 consensus_queue.cc:260] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8ded84feda415bb23bd5a69fc21e4e" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40303 } } peers { permanent_uuid: "e71036b002e54152bf5e6ff0f5095cbd" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 40943 } } peers { permanent_uuid: "288a5face5c14f1aae93eff72286dfbc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 44681 } }
I20260808 18:27:47.538776 15099 heartbeater.cc:503] Master 127.7.164.126:41571 was elected leader, sending a full tablet report...
I20260808 18:27:47.538810 15472 heartbeater.cc:503] Master 127.7.164.126:41571 was elected leader, sending a full tablet report...
I20260808 18:27:47.539316 15498 ts_tablet_manager.cc:1428] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 18:27:47.539665 15296 heartbeater.cc:503] Master 127.7.164.126:41571 was elected leader, sending a full tablet report...
I20260808 18:27:47.540134 15496 ts_tablet_manager.cc:1428] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
W20260808 18:27:47.540416 15297 tablet.cc:2369] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:27:47.589715 15503 raft_consensus.cc:482] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:27:47.589947 15503 raft_consensus.cc:504] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8ded84feda415bb23bd5a69fc21e4e" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40303 } } peers { permanent_uuid: "e71036b002e54152bf5e6ff0f5095cbd" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 40943 } } peers { permanent_uuid: "288a5face5c14f1aae93eff72286dfbc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 44681 } }
I20260808 18:27:47.591158 15503 leader_election.cc:288] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b8ded84feda415bb23bd5a69fc21e4e (127.7.164.66:40303), e71036b002e54152bf5e6ff0f5095cbd (127.7.164.65:40943)
I20260808 18:27:47.593336 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.591466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:40943 (local address 127.7.164.67:37445)
0808 18:27:47.591613 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:47.591632 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:47.591795 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:47.592089 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:47.592092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:47.592107 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:47.592683 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:27:47.592687 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:47.592702 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:27:47.592735 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:47.593102 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:47.593106 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:47.593109 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 18:27:47.593200 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 18:27:47.593233 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 18:27:47.593407 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.591604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:37445 (local address 127.7.164.65:40943)
0808 18:27:47.591827 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:47.591834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:47.591874 (+ 40us) server_negotiation.cc:407] Connection header received
0808 18:27:47.591923 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:47.591926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:47.591972 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:47.592064 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:47.592905 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:47.592910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:47.592913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:47.592929 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:47.592953 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:47.592964 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:47.593090 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:47.593205 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 18:27:47.593254 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:27:47.593585 15505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.591476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:40303 (local address 127.7.164.67:38625)
0808 18:27:47.592046 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:47.592059 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:47.592179 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:47.592475 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:47.592480 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:47.592496 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:47.592932 (+ 436us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:27:47.592935 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:47.592945 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:27:47.592970 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:47.593303 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:47.593306 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:47.593308 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 18:27:47.593394 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 18:27:47.593432 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":126,"thread_start_us":207,"threads_started":1}
I20260808 18:27:47.593648 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.591593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:38625 (local address 127.7.164.66:40303)
0808 18:27:47.591823 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:47.591830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:47.592146 (+ 316us) server_negotiation.cc:407] Connection header received
0808 18:27:47.592297 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:47.592301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:47.592343 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:47.592427 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:47.593115 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:47.593120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:47.593123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:47.593135 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:47.593155 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:47.593164 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:47.593232 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:47.593446 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 18:27:47.593493 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:27:47.594352 15029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e8e1f0a904417a97fee7e2d92e9b3d" candidate_uuid: "288a5face5c14f1aae93eff72286dfbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e71036b002e54152bf5e6ff0f5095cbd" is_pre_election: true
I20260808 18:27:47.594353 15227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e8e1f0a904417a97fee7e2d92e9b3d" candidate_uuid: "288a5face5c14f1aae93eff72286dfbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b8ded84feda415bb23bd5a69fc21e4e" is_pre_election: true
I20260808 18:27:47.594707 15227 raft_consensus.cc:2454] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 288a5face5c14f1aae93eff72286dfbc in term 0.
I20260808 18:27:47.594707 15029 raft_consensus.cc:2454] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 288a5face5c14f1aae93eff72286dfbc in term 0.
I20260808 18:27:47.595248 15320 leader_election.cc:302] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 288a5face5c14f1aae93eff72286dfbc, e71036b002e54152bf5e6ff0f5095cbd; no voters:
I20260808 18:27:47.595544 15503 raft_consensus.cc:2781] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:27:47.595657 15503 raft_consensus.cc:482] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:27:47.595715 15503 raft_consensus.cc:3037] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:47.596809 15503 raft_consensus.cc:504] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8ded84feda415bb23bd5a69fc21e4e" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40303 } } peers { permanent_uuid: "e71036b002e54152bf5e6ff0f5095cbd" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 40943 } } peers { permanent_uuid: "288a5face5c14f1aae93eff72286dfbc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 44681 } }
I20260808 18:27:47.597204 15503 leader_election.cc:288] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [CANDIDATE]: Term 1 election: Requested vote from peers 4b8ded84feda415bb23bd5a69fc21e4e (127.7.164.66:40303), e71036b002e54152bf5e6ff0f5095cbd (127.7.164.65:40943)
I20260808 18:27:47.597632 15227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e8e1f0a904417a97fee7e2d92e9b3d" candidate_uuid: "288a5face5c14f1aae93eff72286dfbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b8ded84feda415bb23bd5a69fc21e4e"
I20260808 18:27:47.597640 15029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e8e1f0a904417a97fee7e2d92e9b3d" candidate_uuid: "288a5face5c14f1aae93eff72286dfbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e71036b002e54152bf5e6ff0f5095cbd"
I20260808 18:27:47.597757 15227 raft_consensus.cc:3037] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:47.597783 15029 raft_consensus.cc:3037] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:47.598984 15227 raft_consensus.cc:2454] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 288a5face5c14f1aae93eff72286dfbc in term 1.
I20260808 18:27:47.599116 15029 raft_consensus.cc:2454] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 288a5face5c14f1aae93eff72286dfbc in term 1.
I20260808 18:27:47.599481 15319 leader_election.cc:302] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 288a5face5c14f1aae93eff72286dfbc, 4b8ded84feda415bb23bd5a69fc21e4e; no voters:
I20260808 18:27:47.600005 15503 raft_consensus.cc:2781] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:47.600274 15503 raft_consensus.cc:686] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [term 1 LEADER]: Becoming Leader. State: Replica: 288a5face5c14f1aae93eff72286dfbc, State: Running, Role: LEADER
I20260808 18:27:47.600641 15503 consensus_queue.cc:237] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8ded84feda415bb23bd5a69fc21e4e" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40303 } } peers { permanent_uuid: "e71036b002e54152bf5e6ff0f5095cbd" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 40943 } } peers { permanent_uuid: "288a5face5c14f1aae93eff72286dfbc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 44681 } }
I20260808 18:27:47.604326 14802 catalog_manager.cc:5495] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc reported cstate change: term changed from 0 to 1, leader changed from <none> to 288a5face5c14f1aae93eff72286dfbc (127.7.164.67). New cstate: current_term: 1 leader_uuid: "288a5face5c14f1aae93eff72286dfbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8ded84feda415bb23bd5a69fc21e4e" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e71036b002e54152bf5e6ff0f5095cbd" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 40943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "288a5face5c14f1aae93eff72286dfbc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 44681 } health_report { overall_health: HEALTHY } } }
W20260808 18:27:47.614897 15100 tablet.cc:2369] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:27:47.615381 15495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.613662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52406 (local address 127.7.164.67:44681)
0808 18:27:47.613799 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:47.613802 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:47.613817 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:27:47.613974 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:47.613976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:47.614048 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:47.614121 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:47.614941 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:47.614944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:47.614946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:47.614957 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:47.614975 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:47.614983 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:47.615094 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:47.615199 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 18:27:47.615242 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:27:47.621845 15029 raft_consensus.cc:1261] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd [term 1 FOLLOWER]: Refusing update from remote peer 288a5face5c14f1aae93eff72286dfbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:47.621837 15227 raft_consensus.cc:1261] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e [term 1 FOLLOWER]: Refusing update from remote peer 288a5face5c14f1aae93eff72286dfbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:47.622710 15509 consensus_queue.cc:1035] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e71036b002e54152bf5e6ff0f5095cbd" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 40943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:47.622980 15503 consensus_queue.cc:1035] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b8ded84feda415bb23bd5a69fc21e4e" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 40303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:47.634158 15516 mvcc.cc:205] Tried to move back new op lower bound from 7316331182569799680 to 7316331182495338496. Current Snapshot: MvccSnapshot[applied={T|T < 7316331182569799680 or (T in {7316331182569799680})}]
I20260808 18:27:47.634410 15514 mvcc.cc:205] Tried to move back new op lower bound from 7316331182569799680 to 7316331182495338496. Current Snapshot: MvccSnapshot[applied={T|T < 7316331182569799680}]
I20260808 18:27:47.640738 14799 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:47.640944 14799 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:47.643473 14799 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb0849da9c1a449997b3ea024633ed66: Sending DeleteTablet for 3 replicas of tablet 14e8e1f0a904417a97fee7e2d92e9b3d
I20260808 18:27:47.644271 15197 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e8e1f0a904417a97fee7e2d92e9b3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:47 UTC) from {username='slave'} at 127.0.0.1:45518
I20260808 18:27:47.644347 15381 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e8e1f0a904417a97fee7e2d92e9b3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:47 UTC) from {username='slave'} at 127.0.0.1:52392
I20260808 18:27:47.644841 15526 tablet_replica.cc:330] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e: stopping tablet replica
I20260808 18:27:47.645279 15526 raft_consensus.cc:2229] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:47.645247 14988 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e8e1f0a904417a97fee7e2d92e9b3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:47 UTC) from {username='slave'} at 127.0.0.1:57782
I20260808 18:27:47.645650 15526 raft_consensus.cc:2258] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:47.645843 15528 tablet_replica.cc:330] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd: stopping tablet replica
I20260808 18:27:47.646242 15528 raft_consensus.cc:2229] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:47.646371 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 14897
I20260808 18:27:47.646646 15528 raft_consensus.cc:2258] T 14e8e1f0a904417a97fee7e2d92e9b3d P e71036b002e54152bf5e6ff0f5095cbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:47.655073 15527 tablet_replica.cc:330] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc: stopping tablet replica
I20260808 18:27:47.655473 15527 raft_consensus.cc:2229] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:27:47.655903 15527 raft_consensus.cc:2258] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:47.656957 15526 ts_tablet_manager.cc:1905] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:47.657593 15527 ts_tablet_manager.cc:1905] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:47.660959 15527 ts_tablet_manager.cc:1918] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:27:47.660955 15526 ts_tablet_manager.cc:1918] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:27:47.661075 15526 log.cc:1196] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/14e8e1f0a904417a97fee7e2d92e9b3d
I20260808 18:27:47.661137 15527 log.cc:1196] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/14e8e1f0a904417a97fee7e2d92e9b3d
I20260808 18:27:47.661435 15526 ts_tablet_manager.cc:1939] T 14e8e1f0a904417a97fee7e2d92e9b3d P 4b8ded84feda415bb23bd5a69fc21e4e: Deleting consensus metadata
I20260808 18:27:47.661556 15527 ts_tablet_manager.cc:1939] T 14e8e1f0a904417a97fee7e2d92e9b3d P 288a5face5c14f1aae93eff72286dfbc: Deleting consensus metadata
I20260808 18:27:47.663197 14785 catalog_manager.cc:4841] TS 4b8ded84feda415bb23bd5a69fc21e4e (127.7.164.66:40303): tablet 14e8e1f0a904417a97fee7e2d92e9b3d (table test-table [id=c14e817c0a9c48b5a1da0db39eeae159]) successfully deleted
W20260808 18:27:47.663326 14786 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:40943 (error 108)
I20260808 18:27:47.663642 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 15102
I20260808 18:27:47.663834 14786 catalog_manager.cc:4841] TS 288a5face5c14f1aae93eff72286dfbc (127.7.164.67:44681): tablet 14e8e1f0a904417a97fee7e2d92e9b3d (table test-table [id=c14e817c0a9c48b5a1da0db39eeae159]) successfully deleted
I20260808 18:27:47.669732 15492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.664114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:40943 (local address 127.0.0.1:57788)
0808 18:27:47.669516 (+ 5402us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:47.669625 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:40943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5283}
W20260808 18:27:47.669976 14786 catalog_manager.cc:4584] TS e71036b002e54152bf5e6ff0f5095cbd (127.7.164.65:40943): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14e8e1f0a904417a97fee7e2d92e9b3d: Network error: Client connection negotiation failed: client connection to 127.7.164.65:40943: connect: Connection refused (error 111)
I20260808 18:27:47.678695 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 15299
I20260808 18:27:47.694175 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 14766
2026-08-08T18:27:47Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1025 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 18:27:47.719362 7825 mini_oidc.cc:152] Starting JWKS server
I20260808 18:27:47.720979 7825 webserver.cc:460] Webserver started at https://127.0.0.1:37627/ using document root <none> and password file <none>
I20260808 18:27:47.721172 7825 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 18:27:47.721457 7825 webserver.cc:460] Webserver started at http://127.0.0.1:38365/ using document root <none> and password file <none>
2026-08-08T18:27:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:47Z Disabled control of system clock
I20260808 18:27:47.731691 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:45417
--webserver_interface=127.7.164.126
--webserver_port=0
--builtin_ntp_servers=127.7.164.84:37295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:45417
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37627/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/testchainca.cert with env {}
W20260808 18:27:47.842355 15537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:47.842618 15537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:47.842658 15537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:47.847011 15537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:47.847072 15537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:47.847088 15537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:47.847105 15537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:47.847154 15537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 18:27:47.847175 15537 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 18:27:47.847200 15537 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37627/jwks.json
I20260808 18:27:47.852658 15537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:37295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:45417
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:45417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37627/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:47.853684 15537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:47.854673 15537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:47.862943 15546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:47.862982 15548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:47.862943 15544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:47.863155 15537 server_base.cc:1034] running on GCE node
I20260808 18:27:47.863794 15537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:47.864835 15537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:47.866044 15537 hybrid_clock.cc:648] HybridClock initialized: now 1786213667866006 us; error 43 us; skew 500 ppm
I20260808 18:27:47.867905 15537 webserver.cc:460] Webserver started at http://127.7.164.126:33111/ using document root <none> and password file <none>
I20260808 18:27:47.868463 15537 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:47.868551 15537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:47.868777 15537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:47.870877 15537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "0687145632ad485b88bbb9ecbc3732fb"
format_stamp: "Formatted at 2026-08-08 18:27:47 on dist-test-slave-jztv"
server_key: "e877b1987924068d2f7b90ffaa1ee1b7"
server_key_iv: "4c723149678fea054df21c979f498393"
server_key_version: "encryptionkey@0"
I20260808 18:27:47.871435 15537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "0687145632ad485b88bbb9ecbc3732fb"
format_stamp: "Formatted at 2026-08-08 18:27:47 on dist-test-slave-jztv"
server_key: "e877b1987924068d2f7b90ffaa1ee1b7"
server_key_iv: "4c723149678fea054df21c979f498393"
server_key_version: "encryptionkey@0"
I20260808 18:27:47.874985 15537 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:27:47.877321 15557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:47.878571 15537 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:47.878706 15537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "0687145632ad485b88bbb9ecbc3732fb"
format_stamp: "Formatted at 2026-08-08 18:27:47 on dist-test-slave-jztv"
server_key: "e877b1987924068d2f7b90ffaa1ee1b7"
server_key_iv: "4c723149678fea054df21c979f498393"
server_key_version: "encryptionkey@0"
I20260808 18:27:47.878813 15537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:47.903098 15537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:47.903890 15537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:47.904107 15537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:47.911010 15621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:45417 every 8 connection(s)
I20260808 18:27:47.911026 15537 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:45417
I20260808 18:27:47.912020 15537 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:47.914858 15623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:47.917850 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 15537
I20260808 18:27:47.917968 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:47.918311 7825 external_mini_cluster.cc:1428] Setting key c25d9bb2530e2ca70551bad58034cb9d
I20260808 18:27:47.920727 15623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb: Bootstrap starting.
I20260808 18:27:47.923034 15623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:47.923755 15623 log.cc:824] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:47.925865 15623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb: No bootstrap required, opened a new log
I20260808 18:27:47.927922 15627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.919684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35976 (local address 127.7.164.126:45417)
0808 18:27:47.920299 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:47.920314 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:47.920349 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:27:47.921050 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:47.921068 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:47.921490 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:47.921815 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:47.922322 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:47.923342 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:47.924344 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:47.924961 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:47.925150 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:47.925897 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:47.925906 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:47.925923 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:47.926443 (+ 520us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:47.926510 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:47.926535 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:47.926634 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:47.926786 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 18:27:47.927049 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":125,"threads_started":1}
I20260808 18:27:47.930269 15623 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0687145632ad485b88bbb9ecbc3732fb" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45417 } }
I20260808 18:27:47.930474 15623 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:47.930553 15623 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0687145632ad485b88bbb9ecbc3732fb, State: Initialized, Role: FOLLOWER
I20260808 18:27:47.930790 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:27:47.931013 15623 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0687145632ad485b88bbb9ecbc3732fb" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45417 } }
I20260808 18:27:47.931159 15623 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:47.931236 15623 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:47.931348 15623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:47.932281 15623 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0687145632ad485b88bbb9ecbc3732fb" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45417 } }
I20260808 18:27:47.932617 15623 leader_election.cc:302] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0687145632ad485b88bbb9ecbc3732fb; no voters:
I20260808 18:27:47.932904 15623 leader_election.cc:288] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:47.933010 15634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:47.933262 15634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [term 1 LEADER]: Becoming Leader. State: Replica: 0687145632ad485b88bbb9ecbc3732fb, State: Running, Role: LEADER
I20260808 18:27:47.933578 15634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0687145632ad485b88bbb9ecbc3732fb" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45417 } }
I20260808 18:27:47.934084 15623 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:47.935361 15638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0687145632ad485b88bbb9ecbc3732fb. Latest consensus state: current_term: 1 leader_uuid: "0687145632ad485b88bbb9ecbc3732fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0687145632ad485b88bbb9ecbc3732fb" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45417 } } }
I20260808 18:27:47.935498 15638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:47.935717 15637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0687145632ad485b88bbb9ecbc3732fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0687145632ad485b88bbb9ecbc3732fb" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45417 } } }
I20260808 18:27:47.935839 15637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:47.935930 15647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:47.939005 15647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:47.944413 15647 catalog_manager.cc:1347] Generated new cluster ID: 707acd89ac3f4c73af19c543c7dd4626
I20260808 18:27:47.944478 15647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:47.952250 15647 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:47.953456 15647 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:47.971830 15647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0687145632ad485b88bbb9ecbc3732fb: Generated new TSK 0
I20260808 18:27:47.972833 15647 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 18:27:48.010879 15671 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 18:27:48.011093 15671 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 18:27:48.011656 15627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:47.991735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35978 (local address 127.7.164.126:45417)
0808 18:27:47.991889 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:47.991894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:47.992057 (+ 163us) server_negotiation.cc:407] Connection header received
0808 18:27:47.992274 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:47.992277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:47.992326 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:47.992413 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0808 18:27:47.992702 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:47.993246 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:47.994481 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:47.994709 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:47.994792 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:47.995165 (+ 373us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 18:27:48.011276 (+ 16111us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37627/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 18:27:48.011529 (+ 253us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35978: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37627/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 18:27:48.011760 15627 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35978: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37627/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 18:27:48.013134 15669 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:47.991593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45417 (local address 127.0.0.1:35978)
0808 18:27:47.991965 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:47.991981 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:47.992119 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:47.992448 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:47.992452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:47.992460 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 18:27:47.992595 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:47.992602 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:47.993351 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:47.993354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:47.994366 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:47.994378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:47.994831 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:47.994836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:47.994925 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:47.994978 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 18:27:48.011545 (+ 16567us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37627/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 18:27:48.011805 (+ 260us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.164.126:45417: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37627/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":107,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.164.126:45417: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37627/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 18:27:48.014281 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 15537
2026-08-08T18:27:48Z chronyd exiting
I20260808 18:27:48.055115 7825 test_util.cc:175] -----------------------------------------------
I20260808 18:27:48.055220 7825 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213640155572-7825-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (337 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 18:27:48.056406 7825 mini_oidc.cc:152] Starting JWKS server
I20260808 18:27:48.057287 7825 webserver.cc:460] Webserver started at https://127.0.0.1:41699/ using document root <none> and password file <none>
I20260808 18:27:48.057423 7825 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 18:27:48.057720 7825 webserver.cc:460] Webserver started at http://127.0.0.1:43057/ using document root <none> and password file <none>
2026-08-08T18:27:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:48Z Disabled control of system clock
I20260808 18:27:48.067948 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:34659
--webserver_interface=127.7.164.126
--webserver_port=0
--builtin_ntp_servers=127.7.164.84:37845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:34659
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41699/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/testchainca.cert with env {}
W20260808 18:27:48.179822 15681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:48.180126 15681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:48.180202 15681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:48.184144 15681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:48.184223 15681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:48.184252 15681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:48.184302 15681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:48.184352 15681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 18:27:48.184387 15681 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 18:27:48.184417 15681 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41699/jwks.json
I20260808 18:27:48.189599 15681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:37845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:34659
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:34659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41699/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:48.190702 15681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:48.191742 15681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:27:48.200314 15681 server_base.cc:1034] running on GCE node
W20260808 18:27:48.200349 15688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:48.200263 15692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:48.200263 15694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:48.201009 15681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:48.202080 15681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:48.203256 15681 hybrid_clock.cc:648] HybridClock initialized: now 1786213668203234 us; error 39 us; skew 500 ppm
I20260808 18:27:48.205035 15681 webserver.cc:460] Webserver started at http://127.7.164.126:37131/ using document root <none> and password file <none>
I20260808 18:27:48.205611 15681 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:48.205698 15681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:48.205920 15681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:48.207945 15681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "c450d15be9d54c68ae2788c96639c0cc"
format_stamp: "Formatted at 2026-08-08 18:27:48 on dist-test-slave-jztv"
server_key: "b6151bb1f946ccbc277961ae8c74a158"
server_key_iv: "c2d9e9958c3fa6db0b61a806c656f0f8"
server_key_version: "encryptionkey@0"
I20260808 18:27:48.208523 15681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "c450d15be9d54c68ae2788c96639c0cc"
format_stamp: "Formatted at 2026-08-08 18:27:48 on dist-test-slave-jztv"
server_key: "b6151bb1f946ccbc277961ae8c74a158"
server_key_iv: "c2d9e9958c3fa6db0b61a806c656f0f8"
server_key_version: "encryptionkey@0"
I20260808 18:27:48.212121 15681 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:27:48.214506 15705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:48.215694 15681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:27:48.215829 15681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "c450d15be9d54c68ae2788c96639c0cc"
format_stamp: "Formatted at 2026-08-08 18:27:48 on dist-test-slave-jztv"
server_key: "b6151bb1f946ccbc277961ae8c74a158"
server_key_iv: "c2d9e9958c3fa6db0b61a806c656f0f8"
server_key_version: "encryptionkey@0"
I20260808 18:27:48.215938 15681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:48.229161 15681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:48.229813 15681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:48.230010 15681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:48.237006 15681 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:34659
I20260808 18:27:48.237016 15782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:34659 every 8 connection(s)
I20260808 18:27:48.237933 15681 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:48.241149 15783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:48.244060 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 15681
I20260808 18:27:48.244205 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:48.244478 7825 external_mini_cluster.cc:1428] Setting key 9c3f319bd36ce6960d534b84a65e8b72
I20260808 18:27:48.247427 15783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc: Bootstrap starting.
I20260808 18:27:48.250190 15783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:48.251046 15783 log.cc:824] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:48.253418 15783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc: No bootstrap required, opened a new log
I20260808 18:27:48.255050 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:48.245866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44958 (local address 127.7.164.126:34659)
0808 18:27:48.246625 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:48.246651 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:48.246696 (+ 45us) server_negotiation.cc:407] Connection header received
0808 18:27:48.247542 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:48.247567 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:48.248081 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:48.248447 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:48.248969 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.250426 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.251365 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.251986 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.252145 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:48.252958 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:48.252964 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:48.252980 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:48.253491 (+ 511us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:48.253564 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:48.253591 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:48.253692 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:48.253804 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 18:27:48.254085 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":153,"threads_started":1}
I20260808 18:27:48.257963 15783 raft_consensus.cc:348] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c450d15be9d54c68ae2788c96639c0cc" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34659 } }
I20260808 18:27:48.258283 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:27:48.258301 15783 raft_consensus.cc:374] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:48.258370 15783 raft_consensus.cc:729] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c450d15be9d54c68ae2788c96639c0cc, State: Initialized, Role: FOLLOWER
I20260808 18:27:48.258824 15783 consensus_queue.cc:260] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c450d15be9d54c68ae2788c96639c0cc" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34659 } }
I20260808 18:27:48.258980 15783 raft_consensus.cc:388] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:48.259047 15783 raft_consensus.cc:482] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:48.259146 15783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:48.260310 15783 raft_consensus.cc:504] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c450d15be9d54c68ae2788c96639c0cc" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34659 } }
I20260808 18:27:48.260669 15783 leader_election.cc:302] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c450d15be9d54c68ae2788c96639c0cc; no voters:
I20260808 18:27:48.260982 15783 leader_election.cc:288] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:48.261132 15789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:48.261348 15789 raft_consensus.cc:686] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [term 1 LEADER]: Becoming Leader. State: Replica: c450d15be9d54c68ae2788c96639c0cc, State: Running, Role: LEADER
I20260808 18:27:48.261634 15789 consensus_queue.cc:237] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c450d15be9d54c68ae2788c96639c0cc" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34659 } }
I20260808 18:27:48.262288 15783 sys_catalog.cc:564] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:48.263307 15792 sys_catalog.cc:455] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c450d15be9d54c68ae2788c96639c0cc. Latest consensus state: current_term: 1 leader_uuid: "c450d15be9d54c68ae2788c96639c0cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c450d15be9d54c68ae2788c96639c0cc" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34659 } } }
I20260808 18:27:48.263321 15790 sys_catalog.cc:455] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c450d15be9d54c68ae2788c96639c0cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c450d15be9d54c68ae2788c96639c0cc" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34659 } } }
I20260808 18:27:48.263806 15790 sys_catalog.cc:458] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:48.263875 15792 sys_catalog.cc:458] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:48.264695 15803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:48.267306 15803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:48.272711 15803 catalog_manager.cc:1347] Generated new cluster ID: 49256888d5f2458f9f632b9864c3c83d
I20260808 18:27:48.272778 15803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:48.287606 15803 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:48.288727 15803 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:48.302043 15803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c450d15be9d54c68ae2788c96639c0cc: Generated new TSK 0
I20260808 18:27:48.302932 15803 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 18:27:48.331861 15822 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 18:27:48.331959 15822 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 18:27:48.332605 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:48.316818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44966 (local address 127.7.164.126:34659)
0808 18:27:48.316962 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:48.316967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:48.317148 (+ 181us) server_negotiation.cc:407] Connection header received
0808 18:27:48.317308 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:48.317310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:48.317356 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:48.317443 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0808 18:27:48.317705 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.318278 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.319549 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.319767 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.319845 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:48.320210 (+ 365us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 18:27:48.332258 (+ 12048us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41699/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 18:27:48.332480 (+ 222us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44966: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41699/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 18:27:48.332687 15786 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44966: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41699/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 18:27:48.332835 15821 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:48.316752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:34659 (local address 127.0.0.1:44966)
0808 18:27:48.317055 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:48.317072 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:48.317190 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:48.317465 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:48.317469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:48.317477 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 18:27:48.317594 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:48.317600 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.318387 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.318391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:48.319435 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:48.319447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.319894 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.319898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:48.319990 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:48.320019 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 18:27:48.332461 (+ 12442us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41699/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 18:27:48.332653 (+ 192us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.164.126:34659: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41699/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":114,"threads_started":1}
I20260808 18:27:48.333568 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 15681
2026-08-08T18:27:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (326 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 18:27:48.382694 7825 mini_oidc.cc:152] Starting JWKS server
I20260808 18:27:48.383598 7825 webserver.cc:460] Webserver started at https://127.0.0.1:39741/ using document root <none> and password file <none>
I20260808 18:27:48.383731 7825 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 18:27:48.384004 7825 webserver.cc:460] Webserver started at http://127.0.0.1:36155/ using document root <none> and password file <none>
2026-08-08T18:27:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:48Z Disabled control of system clock
I20260808 18:27:48.394531 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:39941
--webserver_interface=127.7.164.126
--webserver_port=0
--builtin_ntp_servers=127.7.164.84:42971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:39941
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39741/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 18:27:48.508749 15833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:48.509014 15833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:48.509054 15833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:48.513064 15833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:48.513119 15833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:48.513135 15833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:48.513151 15833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:48.513203 15833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 18:27:48.513238 15833 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 18:27:48.513275 15833 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39741/jwks.json
I20260808 18:27:48.517879 15833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:42971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:39941
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:39941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39741/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:48.519002 15833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:48.520416 15833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:48.528837 15850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:48.528861 15847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:48.528841 15846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:48.529085 15833 server_base.cc:1034] running on GCE node
I20260808 18:27:48.529739 15833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:48.530920 15833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:48.532110 15833 hybrid_clock.cc:648] HybridClock initialized: now 1786213668532093 us; error 28 us; skew 500 ppm
I20260808 18:27:48.533984 15833 webserver.cc:460] Webserver started at http://127.7.164.126:35065/ using document root <none> and password file <none>
I20260808 18:27:48.534662 15833 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:48.534756 15833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:48.534989 15833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:48.537020 15833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "7fdeb610cb274a52841ade85970ed49e"
format_stamp: "Formatted at 2026-08-08 18:27:48 on dist-test-slave-jztv"
server_key: "8058727980d839a254a023fa6b1622ad"
server_key_iv: "7a003bda97e84dc098b68ecd8fe964ea"
server_key_version: "encryptionkey@0"
I20260808 18:27:48.537595 15833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "7fdeb610cb274a52841ade85970ed49e"
format_stamp: "Formatted at 2026-08-08 18:27:48 on dist-test-slave-jztv"
server_key: "8058727980d839a254a023fa6b1622ad"
server_key_iv: "7a003bda97e84dc098b68ecd8fe964ea"
server_key_version: "encryptionkey@0"
I20260808 18:27:48.541234 15833 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:27:48.543758 15859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:48.544930 15833 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:27:48.545065 15833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "7fdeb610cb274a52841ade85970ed49e"
format_stamp: "Formatted at 2026-08-08 18:27:48 on dist-test-slave-jztv"
server_key: "8058727980d839a254a023fa6b1622ad"
server_key_iv: "7a003bda97e84dc098b68ecd8fe964ea"
server_key_version: "encryptionkey@0"
I20260808 18:27:48.545176 15833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:48.567662 15833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:48.568385 15833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:48.568583 15833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:48.575698 15833 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:39941
I20260808 18:27:48.575713 15925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:39941 every 8 connection(s)
I20260808 18:27:48.576643 15833 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:48.579814 15932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:48.580744 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 15833
I20260808 18:27:48.580870 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:48.581175 7825 external_mini_cluster.cc:1428] Setting key aa725853aaf213887e8a09d0413c0887
I20260808 18:27:48.585955 15932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e: Bootstrap starting.
I20260808 18:27:48.588689 15932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:48.589457 15932 log.cc:824] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:48.590826 15935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:48.582557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54626 (local address 127.7.164.126:39941)
0808 18:27:48.583193 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:48.583208 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:48.583243 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:27:48.583880 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:48.583896 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:48.584343 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:48.584666 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:48.585093 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.586013 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.587253 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.587750 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.587895 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:48.588843 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:48.588851 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:48.588869 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:48.589261 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:48.589316 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:48.589339 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:48.589423 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:48.589667 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 18:27:48.589962 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":138,"threads_started":1}
I20260808 18:27:48.591615 15932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e: No bootstrap required, opened a new log
I20260808 18:27:48.593622 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:27:48.595592 15932 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fdeb610cb274a52841ade85970ed49e" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 39941 } }
I20260808 18:27:48.595840 15932 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:48.595917 15932 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fdeb610cb274a52841ade85970ed49e, State: Initialized, Role: FOLLOWER
I20260808 18:27:48.596371 15932 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fdeb610cb274a52841ade85970ed49e" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 39941 } }
I20260808 18:27:48.596534 15932 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:48.596617 15932 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:48.596719 15932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:48.597649 15932 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fdeb610cb274a52841ade85970ed49e" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 39941 } }
I20260808 18:27:48.597993 15932 leader_election.cc:302] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fdeb610cb274a52841ade85970ed49e; no voters:
I20260808 18:27:48.598390 15932 leader_election.cc:288] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:48.598521 15937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:48.598764 15937 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [term 1 LEADER]: Becoming Leader. State: Replica: 7fdeb610cb274a52841ade85970ed49e, State: Running, Role: LEADER
I20260808 18:27:48.599095 15937 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fdeb610cb274a52841ade85970ed49e" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 39941 } }
I20260808 18:27:48.599704 15932 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:48.600826 15939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fdeb610cb274a52841ade85970ed49e. Latest consensus state: current_term: 1 leader_uuid: "7fdeb610cb274a52841ade85970ed49e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fdeb610cb274a52841ade85970ed49e" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 39941 } } }
I20260808 18:27:48.600970 15939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:48.601356 15948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:48.601526 15938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fdeb610cb274a52841ade85970ed49e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fdeb610cb274a52841ade85970ed49e" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 39941 } } }
I20260808 18:27:48.601648 15938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:48.604164 15948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:48.610713 15948 catalog_manager.cc:1347] Generated new cluster ID: e5df62ea4917446185c9a75d135ccadf
I20260808 18:27:48.610793 15948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:48.625546 15948 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:48.626556 15948 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:48.637257 15948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fdeb610cb274a52841ade85970ed49e: Generated new TSK 0
I20260808 18:27:48.638041 15948 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:27:48.652211 15935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:48.646870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54642 (local address 127.7.164.126:39941)
0808 18:27:48.647014 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:48.647020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:48.647232 (+ 212us) server_negotiation.cc:407] Connection header received
0808 18:27:48.647444 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:48.647447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:48.647500 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:48.647601 (+ 101us) server_negotiation.cc:226] Negotiated authn=JWT
0808 18:27:48.647880 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.648501 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.649720 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.649996 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.650122 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:48.650441 (+ 319us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 18:27:48.651848 (+ 1407us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39741/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 18:27:48.652072 (+ 224us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54642: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39741/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 18:27:48.652283 15935 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54642: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39741/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 18:27:48.654048 15969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:48.646761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:39941 (local address 127.0.0.1:54642)
0808 18:27:48.647133 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:48.647148 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:48.647274 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:48.647627 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:48.647630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:48.647639 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 18:27:48.647769 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:48.647775 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.648623 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.648627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:48.649606 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:48.649617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.650116 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.650120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:48.650235 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:48.650268 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 18:27:48.652028 (+ 1760us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39741/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 18:27:48.653862 (+ 1834us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.164.126:39941: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39741/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":1626,"thread_start_us":128,"threads_started":1}
I20260808 18:27:48.654661 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 15833
E20260808 18:27:48.673856 15970 webserver.cc:533] Webserver: sslize failed:
E20260808 18:27:48.673967 15970 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T18:27:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (323 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T18:27:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:48Z Disabled control of system clock
I20260808 18:27:48.716126 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:44021
--webserver_interface=127.7.164.126
--webserver_port=0
--builtin_ntp_servers=127.7.164.84:38825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:44021
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/testchainca.cert with env {}
W20260808 18:27:48.830070 15975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:48.830412 15975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:48.830482 15975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:48.834486 15975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:48.834540 15975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:48.834555 15975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:48.834573 15975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:48.834587 15975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 18:27:48.834602 15975 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 18:27:48.834656 15975 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 18:27:48.840052 15975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:44021
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:44021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:48.841146 15975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:48.842171 15975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:48.850909 15983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:48.850909 15987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:48.850946 15984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:48.851233 15975 server_base.cc:1034] running on GCE node
I20260808 18:27:48.851800 15975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:48.852856 15975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:48.854063 15975 hybrid_clock.cc:648] HybridClock initialized: now 1786213668854049 us; error 30 us; skew 500 ppm
I20260808 18:27:48.855973 15975 webserver.cc:460] Webserver started at http://127.7.164.126:42231/ using document root <none> and password file <none>
I20260808 18:27:48.856552 15975 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:48.856616 15975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:48.856869 15975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:48.859022 15975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "e3356a52b8dd4f70a5509d1c2810796d"
format_stamp: "Formatted at 2026-08-08 18:27:48 on dist-test-slave-jztv"
server_key: "d62be2e3db0b0858c86c986009bb7196"
server_key_iv: "b4b19ae68d16d13d1857ea4a2691b1ae"
server_key_version: "encryptionkey@0"
I20260808 18:27:48.859629 15975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "e3356a52b8dd4f70a5509d1c2810796d"
format_stamp: "Formatted at 2026-08-08 18:27:48 on dist-test-slave-jztv"
server_key: "d62be2e3db0b0858c86c986009bb7196"
server_key_iv: "b4b19ae68d16d13d1857ea4a2691b1ae"
server_key_version: "encryptionkey@0"
I20260808 18:27:48.863415 15975 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:27:48.865868 15999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:48.867086 15975 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:48.867221 15975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "e3356a52b8dd4f70a5509d1c2810796d"
format_stamp: "Formatted at 2026-08-08 18:27:48 on dist-test-slave-jztv"
server_key: "d62be2e3db0b0858c86c986009bb7196"
server_key_iv: "b4b19ae68d16d13d1857ea4a2691b1ae"
server_key_version: "encryptionkey@0"
I20260808 18:27:48.867328 15975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:48.896685 15975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:48.897433 15975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:48.897644 15975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:48.904856 15975 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:44021
I20260808 18:27:48.904881 16082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:44021 every 8 connection(s)
I20260808 18:27:48.905821 15975 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:48.909080 16083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:48.912621 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 15975
I20260808 18:27:48.912736 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:48.913046 7825 external_mini_cluster.cc:1428] Setting key fc01c8c9f1212272e246b24a23915bbc
I20260808 18:27:48.915498 16083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d: Bootstrap starting.
I20260808 18:27:48.918439 16083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:48.919358 16083 log.cc:824] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:48.921734 16083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d: No bootstrap required, opened a new log
I20260808 18:27:48.924016 16089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:48.914517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37530 (local address 127.7.164.126:44021)
0808 18:27:48.915213 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:48.915239 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:48.915284 (+ 45us) server_negotiation.cc:407] Connection header received
0808 18:27:48.916112 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:48.916136 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:48.916651 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:48.917033 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:48.917489 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.919057 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.920129 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.920828 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.920993 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:48.921855 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:48.921862 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:48.921878 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:27:48.922427 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:48.922500 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:27:48.922527 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:48.922640 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:48.922802 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 18:27:48.923074 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":106,"threads_started":1}
I20260808 18:27:48.926369 16083 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3356a52b8dd4f70a5509d1c2810796d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44021 } }
I20260808 18:27:48.926666 16083 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:48.926744 16083 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3356a52b8dd4f70a5509d1c2810796d, State: Initialized, Role: FOLLOWER
I20260808 18:27:48.927098 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:27:48.927235 16083 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3356a52b8dd4f70a5509d1c2810796d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44021 } }
I20260808 18:27:48.927393 16083 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:48.927464 16083 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:48.927560 16083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:48.928619 16083 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3356a52b8dd4f70a5509d1c2810796d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44021 } }
I20260808 18:27:48.928975 16083 leader_election.cc:302] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3356a52b8dd4f70a5509d1c2810796d; no voters:
I20260808 18:27:48.929258 16083 leader_election.cc:288] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:48.929355 16094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:48.929625 16094 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [term 1 LEADER]: Becoming Leader. State: Replica: e3356a52b8dd4f70a5509d1c2810796d, State: Running, Role: LEADER
I20260808 18:27:48.929939 16094 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3356a52b8dd4f70a5509d1c2810796d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44021 } }
I20260808 18:27:48.930502 16083 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:48.931113 16094 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3356a52b8dd4f70a5509d1c2810796d. Latest consensus state: current_term: 1 leader_uuid: "e3356a52b8dd4f70a5509d1c2810796d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3356a52b8dd4f70a5509d1c2810796d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44021 } } }
I20260808 18:27:48.931236 16094 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:48.931985 16096 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3356a52b8dd4f70a5509d1c2810796d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3356a52b8dd4f70a5509d1c2810796d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44021 } } }
I20260808 18:27:48.932106 16096 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:48.933017 16105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:48.935822 16105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:48.941480 16105 catalog_manager.cc:1347] Generated new cluster ID: aefe2f31cd39467da6c6ed9829511181
I20260808 18:27:48.941555 16105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:48.956550 16105 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:48.957796 16105 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:48.962968 16105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3356a52b8dd4f70a5509d1c2810796d: Generated new TSK 0
I20260808 18:27:48.963701 16105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:27:49.063482 16089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:48.977440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37536 (local address 127.7.164.126:44021)
0808 18:27:48.977566 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:48.977572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:48.977763 (+ 191us) server_negotiation.cc:407] Connection header received
0808 18:27:48.977957 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:48.977960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:48.978007 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:48.978128 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0808 18:27:48.978389 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.978934 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.979833 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.980055 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.980174 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:48.980481 (+ 307us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 18:27:49.062981 (+ 82500us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 18:27:49.063317 (+ 336us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37536: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 18:27:49.063594 16089 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37536: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 18:27:49.063617 16131 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:48.977339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:44021 (local address 127.0.0.1:37536)
0808 18:27:48.977675 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:48.977687 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:48.977816 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:48.978128 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:48.978131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:48.978140 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 18:27:48.978271 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:48.978278 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.979055 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.979059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:48.979701 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:48.979709 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:48.980181 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:48.980185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:48.980274 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:48.980303 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 18:27:49.063218 (+ 82915us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 18:27:49.063420 (+ 202us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.164.126:44021: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":111,"threads_started":1}
I20260808 18:27:49.064457 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 15975
2026-08-08T18:27:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (385 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (128 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (128 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:49 dist-test-slave-jztv krb5kdc[16147](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:49 dist-test-slave-jztv krb5kdc[16147](info): set up 2 sockets
Aug 08 18:27:49 dist-test-slave-jztv krb5kdc[16147](info): commencing operation
krb5kdc: starting...
W20260808 18:27:50.561412 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:27:50 dist-test-slave-jztv krb5kdc[16147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:27:50.697105 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:40011
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:42239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:40011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:50.812798 16169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:50.813158 16169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:50.813231 16169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:50.817400 16169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:50.817512 16169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:50.817548 16169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:50.817584 16169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:50.817636 16169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:50.823294 16169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:42239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:40011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:40011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:50.824477 16169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:50.825583 16169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:50.834192 16179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:50.834196 16180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:50.834192 16182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:50.834550 16169 server_base.cc:1034] running on GCE node
I20260808 18:27:50.835110 16169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:50.836171 16169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:50.837334 16169 hybrid_clock.cc:648] HybridClock initialized: now 1786213670837297 us; error 52 us; skew 500 ppm
Aug 08 18:27:50 dist-test-slave-jztv krb5kdc[16147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213670, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:50.840135 16169 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:27:50.841032 16169 webserver.cc:460] Webserver started at http://127.7.164.126:34811/ using document root <none> and password file <none>
I20260808 18:27:50.841598 16169 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:50.841686 16169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:50.841920 16169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:50.844067 16169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "be5adb2086ff464ba8dd156dd2c5e547"
format_stamp: "Formatted at 2026-08-08 18:27:50 on dist-test-slave-jztv"
server_key: "60d48067cca4a762b88046651e29ca36"
server_key_iv: "8be8aad587dda22a9fdf77faa958c554"
server_key_version: "encryptionkey@0"
I20260808 18:27:50.844794 16169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "be5adb2086ff464ba8dd156dd2c5e547"
format_stamp: "Formatted at 2026-08-08 18:27:50 on dist-test-slave-jztv"
server_key: "60d48067cca4a762b88046651e29ca36"
server_key_iv: "8be8aad587dda22a9fdf77faa958c554"
server_key_version: "encryptionkey@0"
I20260808 18:27:50.848835 16169 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:27:50.851308 16189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:50.852766 16169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:27:50.852905 16169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "be5adb2086ff464ba8dd156dd2c5e547"
format_stamp: "Formatted at 2026-08-08 18:27:50 on dist-test-slave-jztv"
server_key: "60d48067cca4a762b88046651e29ca36"
server_key_iv: "8be8aad587dda22a9fdf77faa958c554"
server_key_version: "encryptionkey@0"
I20260808 18:27:50.853029 16169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:50.870532 16169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:50.873823 16169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:50.873979 16169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:50.880820 16169 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:40011
I20260808 18:27:50.880903 16265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:40011 every 8 connection(s)
I20260808 18:27:50.881808 16169 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:50.884589 16266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:50.884783 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 16169
I20260808 18:27:50.884922 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:50.885205 7825 external_mini_cluster.cc:1428] Setting key 4afeaa4de68e8d4892aa6c4f3403e01c
I20260808 18:27:50.890933 16266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547: Bootstrap starting.
I20260808 18:27:50.893463 16266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547: Neither blocks nor log segments found. Creating new log.
Aug 08 18:27:50 dist-test-slave-jztv krb5kdc[16147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:50.894312 16266 log.cc:824] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:50.896358 16266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547: No bootstrap required, opened a new log
I20260808 18:27:50.899487 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:50.886639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32832 (local address 127.7.164.126:40011)
0808 18:27:50.887059 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:50.887074 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:50.887110 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:27:50.887810 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:50.887834 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:50.888147 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:50.888479 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:50.889263 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:50.890312 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:50.891181 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:50.891862 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:50.892046 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:50.894714 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:50.894722 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:50.894740 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:50.894774 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:50.896613 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:50.897085 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:50.897090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:50.897095 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:50.897213 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:50.897511 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:50.897515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:50.897516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:50.897924 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:50.898133 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:50.898454 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 18:27:50.898760 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":133,"threads_started":1}
I20260808 18:27:50.900170 16266 raft_consensus.cc:348] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5adb2086ff464ba8dd156dd2c5e547" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40011 } }
I20260808 18:27:50.900424 16266 raft_consensus.cc:374] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:50.900519 16266 raft_consensus.cc:729] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be5adb2086ff464ba8dd156dd2c5e547, State: Initialized, Role: FOLLOWER
I20260808 18:27:50.900993 16266 consensus_queue.cc:260] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5adb2086ff464ba8dd156dd2c5e547" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40011 } }
I20260808 18:27:50.901144 16266 raft_consensus.cc:388] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:50.901201 16266 raft_consensus.cc:482] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:50.901284 16266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:50.902429 16266 raft_consensus.cc:504] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5adb2086ff464ba8dd156dd2c5e547" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40011 } }
I20260808 18:27:50.902777 16266 leader_election.cc:302] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be5adb2086ff464ba8dd156dd2c5e547; no voters:
I20260808 18:27:50.903107 16266 leader_election.cc:288] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:50.903230 16272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:50.903460 16272 raft_consensus.cc:686] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [term 1 LEADER]: Becoming Leader. State: Replica: be5adb2086ff464ba8dd156dd2c5e547, State: Running, Role: LEADER
I20260808 18:27:50.903777 16272 consensus_queue.cc:237] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5adb2086ff464ba8dd156dd2c5e547" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40011 } }
I20260808 18:27:50.904331 16266 sys_catalog.cc:564] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:50.905470 16273 sys_catalog.cc:455] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be5adb2086ff464ba8dd156dd2c5e547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5adb2086ff464ba8dd156dd2c5e547" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40011 } } }
I20260808 18:27:50.905606 16273 sys_catalog.cc:458] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:50.905894 16277 sys_catalog.cc:455] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be5adb2086ff464ba8dd156dd2c5e547. Latest consensus state: current_term: 1 leader_uuid: "be5adb2086ff464ba8dd156dd2c5e547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be5adb2086ff464ba8dd156dd2c5e547" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40011 } } }
I20260808 18:27:50.905993 16277 sys_catalog.cc:458] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:50.906198 16290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:50.909377 16290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:50.915222 16290 catalog_manager.cc:1347] Generated new cluster ID: 27cef048e8b0459e82482e5589b3a6b6
I20260808 18:27:50.915295 16290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:50.923993 16290 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:50.924990 16290 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:50.931849 16290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be5adb2086ff464ba8dd156dd2c5e547: Generated new TSK 0
I20260808 18:27:50.932507 16290 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:27:50.963153 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:40011
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:42239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:51.086371 16303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:51.086658 16303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:51.086732 16303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:51.090921 16303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:51.090984 16303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:51.091076 16303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:27:51.097530 16303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:42239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:40011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:51.098932 16303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:51.100092 16303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:51.110270 16315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:51.110210 16311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:51.110395 16303 server_base.cc:1034] running on GCE node
W20260808 18:27:51.110316 16312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:51.110986 16303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:51.111701 16303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:51.112891 16303 hybrid_clock.cc:648] HybridClock initialized: now 1786213671112870 us; error 31 us; skew 500 ppm
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:51.115806 16303 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:27:51.116758 16303 webserver.cc:460] Webserver started at http://127.7.164.65:43735/ using document root <none> and password file <none>
I20260808 18:27:51.117314 16303 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:51.117376 16303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:51.117570 16303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:51.119776 16303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "88dbaac281384f838837d4159244be6f"
format_stamp: "Formatted at 2026-08-08 18:27:51 on dist-test-slave-jztv"
server_key: "f0eabb79817f5e66dfc9e0615ebe2c0c"
server_key_iv: "7cd7a21ed60ed2ef0e612164e3aea415"
server_key_version: "encryptionkey@0"
I20260808 18:27:51.120332 16303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "88dbaac281384f838837d4159244be6f"
format_stamp: "Formatted at 2026-08-08 18:27:51 on dist-test-slave-jztv"
server_key: "f0eabb79817f5e66dfc9e0615ebe2c0c"
server_key_iv: "7cd7a21ed60ed2ef0e612164e3aea415"
server_key_version: "encryptionkey@0"
I20260808 18:27:51.124403 16303 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:27:51.127050 16324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:51.128362 16303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:27:51.128496 16303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "88dbaac281384f838837d4159244be6f"
format_stamp: "Formatted at 2026-08-08 18:27:51 on dist-test-slave-jztv"
server_key: "f0eabb79817f5e66dfc9e0615ebe2c0c"
server_key_iv: "7cd7a21ed60ed2ef0e612164e3aea415"
server_key_version: "encryptionkey@0"
I20260808 18:27:51.128593 16303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:51.139942 16303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:51.143038 16303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:51.143249 16303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:51.143874 16303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:51.144762 16303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:51.144835 16303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:51.144903 16303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:51.144951 16303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:51.154316 16303 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:46053
I20260808 18:27:51.154323 16480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:46053 every 8 connection(s)
I20260808 18:27:51.155289 16303 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:27:51.160590 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 16303
I20260808 18:27:51.160691 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:27:51.160992 7825 external_mini_cluster.cc:1428] Setting key dac09153ab55744cf5e3ca4b74940626
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:51.171164 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:51.157155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:55321 (local address 127.7.164.126:40011)
0808 18:27:51.157316 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:51.157321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:51.158233 (+ 912us) server_negotiation.cc:407] Connection header received
0808 18:27:51.159671 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:51.159675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:51.159737 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:51.159853 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:51.161199 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.161929 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.163669 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.163897 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.163982 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:51.166963 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:51.166966 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:51.166969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:51.167002 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:51.168841 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:51.169379 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:51.169385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:51.169388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:51.169500 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:51.169843 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:51.169849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:51.169853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:51.170123 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:51.170244 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:51.170804 (+ 560us) server_negotiation.cc:299] Negotiation successful
0808 18:27:51.170952 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:27:51.171854 16489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:51.157616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:40011 (local address 127.7.164.65:55321)
0808 18:27:51.158074 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:51.158118 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:51.159446 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:51.160029 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:51.160036 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:51.160404 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:51.160985 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:51.161001 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.162054 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.162057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:51.163537 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:51.163554 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.164052 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.164059 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:51.164300 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:51.165062 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:51.165083 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:51.166794 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:51.168985 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:51.168990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:51.169003 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:51.169258 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:51.169603 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:51.169605 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:51.169606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:51.169735 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:51.170288 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:51.170294 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:51.170538 (+ 244us) client_negotiation.cc:769] Sending connection context
0808 18:27:51.170690 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 18:27:51.170958 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:27:51.173148 16486 heartbeater.cc:347] Connected to a master server at 127.7.164.126:40011
I20260808 18:27:51.173368 16486 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:51.173888 16486 heartbeater.cc:511] Master 127.7.164.126:40011 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
I20260808 18:27:51.175725 16204 ts_manager.cc:194] Registered new tserver with Master: 88dbaac281384f838837d4159244be6f (127.7.164.65:46053)
I20260808 18:27:51.176864 16204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:55321
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:27:51.214164 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:40011
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:42239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:51.329571 16493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:51.329872 16493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:51.329950 16493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:51.334108 16493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:51.334185 16493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:51.334288 16493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:27:51.339526 16493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:42239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:40011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:51.340581 16493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:51.341598 16493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:51.351130 16501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:51.351150 16503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:51.351150 16500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:51.351337 16493 server_base.cc:1034] running on GCE node
I20260808 18:27:51.351879 16493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:51.352558 16493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:51.353744 16493 hybrid_clock.cc:648] HybridClock initialized: now 1786213671353711 us; error 46 us; skew 500 ppm
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:51.356532 16493 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:27:51.357501 16493 webserver.cc:460] Webserver started at http://127.7.164.66:36037/ using document root <none> and password file <none>
I20260808 18:27:51.358132 16493 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:51.358247 16493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:51.358493 16493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:51.360601 16493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "a82df8b1a2b742cba3ca228a5ae7d4e3"
format_stamp: "Formatted at 2026-08-08 18:27:51 on dist-test-slave-jztv"
server_key: "08e7d7e4fdb422665b31155edb804ccb"
server_key_iv: "d8de68b7c8784ded22b70c4af1ef2aa1"
server_key_version: "encryptionkey@0"
I20260808 18:27:51.361238 16493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "a82df8b1a2b742cba3ca228a5ae7d4e3"
format_stamp: "Formatted at 2026-08-08 18:27:51 on dist-test-slave-jztv"
server_key: "08e7d7e4fdb422665b31155edb804ccb"
server_key_iv: "d8de68b7c8784ded22b70c4af1ef2aa1"
server_key_version: "encryptionkey@0"
I20260808 18:27:51.364974 16493 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260808 18:27:51.367467 16518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:51.368633 16493 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:27:51.368768 16493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "a82df8b1a2b742cba3ca228a5ae7d4e3"
format_stamp: "Formatted at 2026-08-08 18:27:51 on dist-test-slave-jztv"
server_key: "08e7d7e4fdb422665b31155edb804ccb"
server_key_iv: "d8de68b7c8784ded22b70c4af1ef2aa1"
server_key_version: "encryptionkey@0"
I20260808 18:27:51.368877 16493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:51.382014 16493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:51.384840 16493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:51.385037 16493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:51.385613 16493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:51.386493 16493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:51.386561 16493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:51.386626 16493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:51.386667 16493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:51.395655 16493 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:43003
I20260808 18:27:51.395686 16690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:43003 every 8 connection(s)
I20260808 18:27:51.396559 16493 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:27:51.401876 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 16493
I20260808 18:27:51.402004 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:27:51.402339 7825 external_mini_cluster.cc:1428] Setting key 22cdfdced79e084c711b3f74f1aa66e1
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:51.411796 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:51.398436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:36357 (local address 127.7.164.126:40011)
0808 18:27:51.398603 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:51.398609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:51.399616 (+ 1007us) server_negotiation.cc:407] Connection header received
0808 18:27:51.400935 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:51.400938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:51.400988 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:51.401070 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:51.402307 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.403012 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.404420 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.404808 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.404932 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:51.408049 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:51.408053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:51.408056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:51.408091 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:51.409583 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:51.410181 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:51.410184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:51.410185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:51.410312 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:51.410666 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:51.410669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:51.410671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:51.410864 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:51.410972 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:51.411480 (+ 508us) server_negotiation.cc:299] Negotiation successful
0808 18:27:51.411618 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:27:51.412597 16693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:51.398877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:40011 (local address 127.7.164.66:36357)
0808 18:27:51.399440 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:51.399496 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:51.400750 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:51.401223 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:51.401229 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:51.401592 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:51.402134 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:51.402151 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.403156 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.403159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:51.404273 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:51.404283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.404977 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.404982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:51.405203 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:51.405885 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:51.405915 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:51.407880 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:51.409708 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:51.409714 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:51.409730 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:51.410065 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:51.410404 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:51.410408 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:51.410410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:51.410557 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:51.410972 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:51.410980 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:51.411237 (+ 257us) client_negotiation.cc:769] Sending connection context
0808 18:27:51.411420 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 18:27:51.411760 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":154,"threads_started":1}
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
I20260808 18:27:51.413888 16691 heartbeater.cc:347] Connected to a master server at 127.7.164.126:40011
I20260808 18:27:51.414117 16691 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:51.414657 16691 heartbeater.cc:511] Master 127.7.164.126:40011 requested a full tablet report, sending...
I20260808 18:27:51.415709 16204 ts_manager.cc:194] Registered new tserver with Master: a82df8b1a2b742cba3ca228a5ae7d4e3 (127.7.164.66:43003)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
I20260808 18:27:51.416493 16204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:36357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:27:51.433534 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:40011
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:42239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:51.548635 16700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:51.548877 16700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:51.548918 16700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:51.552989 16700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:51.553048 16700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:51.553104 16700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:27:51.558491 16700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:42239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:40011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:51.559573 16700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:51.560590 16700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:51.570312 16708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:51.570396 16713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:51.570312 16707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:51.570366 16700 server_base.cc:1034] running on GCE node
I20260808 18:27:51.570911 16700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:51.571493 16700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:51.572652 16700 hybrid_clock.cc:648] HybridClock initialized: now 1786213671572635 us; error 26 us; skew 500 ppm
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:51.575439 16700 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:27:51.576241 16700 webserver.cc:460] Webserver started at http://127.7.164.67:43297/ using document root <none> and password file <none>
I20260808 18:27:51.576754 16700 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:51.576812 16700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:51.576977 16700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:51.579015 16700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "76f7ac2117d84355870d9c06a3439985"
format_stamp: "Formatted at 2026-08-08 18:27:51 on dist-test-slave-jztv"
server_key: "66c5a3ce02471dd7fff19fb4e7b55f82"
server_key_iv: "2dc03e9fba75f0f7efa0af67bf7619c7"
server_key_version: "encryptionkey@0"
I20260808 18:27:51.579550 16700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "76f7ac2117d84355870d9c06a3439985"
format_stamp: "Formatted at 2026-08-08 18:27:51 on dist-test-slave-jztv"
server_key: "66c5a3ce02471dd7fff19fb4e7b55f82"
server_key_iv: "2dc03e9fba75f0f7efa0af67bf7619c7"
server_key_version: "encryptionkey@0"
I20260808 18:27:51.583338 16700 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:27:51.585856 16727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:51.587142 16700 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:51.587281 16700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "76f7ac2117d84355870d9c06a3439985"
format_stamp: "Formatted at 2026-08-08 18:27:51 on dist-test-slave-jztv"
server_key: "66c5a3ce02471dd7fff19fb4e7b55f82"
server_key_iv: "2dc03e9fba75f0f7efa0af67bf7619c7"
server_key_version: "encryptionkey@0"
I20260808 18:27:51.587399 16700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:51.604231 16700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:51.607841 16700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:51.608069 16700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:51.608719 16700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:51.609615 16700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:51.609683 16700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:51.609753 16700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:51.609798 16700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:51.619357 16700 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:40653
I20260808 18:27:51.619377 16886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:40653 every 8 connection(s)
I20260808 18:27:51.620262 16700 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:27:51.620867 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 16700
I20260808 18:27:51.621019 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:27:51.621317 7825 external_mini_cluster.cc:1428] Setting key 4cef89e4286d37fdd5dbb59ecd9f75a8
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:27:51.634316 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:51.622207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:36697 (local address 127.7.164.126:40011)
0808 18:27:51.622374 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:51.622378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:51.623472 (+ 1094us) server_negotiation.cc:407] Connection header received
0808 18:27:51.624898 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:51.624901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:51.624951 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:51.625026 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:51.626133 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.626747 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.627867 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.628091 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.628175 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:51.630609 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:51.630612 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:51.630614 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:51.630640 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:51.632143 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:51.632670 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:51.632673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:51.632674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:51.632753 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:51.633129 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:51.633132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:51.633133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:51.633354 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:51.633443 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:51.633993 (+ 550us) server_negotiation.cc:299] Negotiation successful
0808 18:27:51.634099 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:27:51.635390 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:51.622738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:40011 (local address 127.7.164.67:36697)
0808 18:27:51.623276 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:51.623335 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:51.624702 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:51.625157 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:51.625163 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:51.625513 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:51.626013 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:51.626027 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.626873 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.626876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:51.627751 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:51.627761 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.628218 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.628221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:51.628361 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:51.628919 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:51.628937 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:51.630424 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:51.632256 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:51.632260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:51.632272 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:51.632564 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:51.632862 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:51.632865 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:51.632866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:51.633013 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:51.633468 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:51.633473 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:51.633715 (+ 242us) client_negotiation.cc:769] Sending connection context
0808 18:27:51.633954 (+ 239us) client_negotiation.cc:240] Negotiation successful
0808 18:27:51.634313 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":113,"threads_started":1}
I20260808 18:27:51.636489 16887 heartbeater.cc:347] Connected to a master server at 127.7.164.126:40011
I20260808 18:27:51.636772 16887 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:51.637290 16887 heartbeater.cc:511] Master 127.7.164.126:40011 requested a full tablet report, sending...
I20260808 18:27:51.638348 16204 ts_manager.cc:194] Registered new tserver with Master: 76f7ac2117d84355870d9c06a3439985 (127.7.164.67:40653)
I20260808 18:27:51.638878 16204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:36697
I20260808 18:27:51.647643 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/test-admin.keytab.
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:51.667325 7825 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 18:27:51.670538 16897 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:51.668926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50950 (local address 127.7.164.65:46053)
0808 18:27:51.669287 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:51.669312 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:51.669337 (+ 25us) server_negotiation.cc:407] Connection header received
0808 18:27:51.669391 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:51.669396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:51.669601 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:51.669731 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:51.670334 (+ 603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50950: BlockingRecv error: recv got EOF from 127.0.0.1:50950 (error 108)
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":96,"threads_started":1}
W20260808 18:27:51.670614 7825 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/krb5cc' of type FILE
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:51.671698 7825 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:51.672874 7825 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.65@KRBTEST.COM
I20260808 18:27:51.682348 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:51.673878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50952 (local address 127.7.164.65:46053)
0808 18:27:51.674044 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:51.674048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:51.674085 (+ 37us) server_negotiation.cc:407] Connection header received
0808 18:27:51.674201 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:51.674203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:51.674287 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:51.674357 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:51.674866 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.675636 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.676426 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.676811 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.676898 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:51.679191 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:51.679200 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:51.679214 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:51.679238 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:51.680671 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:51.681078 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:51.681083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:51.681084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:51.681164 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:51.681391 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:51.681393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:51.681394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:51.681633 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:51.681795 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:51.681954 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 18:27:51.682141 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 18:27:51.686172 16897 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:51.685161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50964 (local address 127.7.164.65:46053)
0808 18:27:51.685292 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:51.685297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:51.685378 (+ 81us) server_negotiation.cc:407] Connection header received
0808 18:27:51.685530 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:51.685533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:51.685575 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:51.685660 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:51.686056 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50964: BlockingRecv error: recv got EOF from 127.0.0.1:50964 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 18:27:51.686394 7825 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/krb5cc' of type FILE
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:51.687378 7825 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:51.688217 7825 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.65@KRBTEST.COM
I20260808 18:27:51.697227 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:51.689009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50970 (local address 127.7.164.65:46053)
0808 18:27:51.689127 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:51.689130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:51.689316 (+ 186us) server_negotiation.cc:407] Connection header received
0808 18:27:51.689387 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:51.689389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:51.689424 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:51.689487 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:51.689928 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.690475 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.691431 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.691755 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.691865 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:51.694041 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:51.694045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:51.694047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:51.694094 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:51.695706 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:51.696100 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:51.696102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:51.696103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:51.696177 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:51.696437 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:51.696440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:51.696442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:51.696627 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:51.696759 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:51.696909 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 18:27:51.697004 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 18:27:51.699823 16897 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:51.698989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50984 (local address 127.7.164.65:46053)
0808 18:27:51.699111 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:51.699114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:51.699233 (+ 119us) server_negotiation.cc:407] Connection header received
0808 18:27:51.699314 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:51.699316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:51.699359 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:51.699424 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:51.699729 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50984: BlockingRecv error: recv got EOF from 127.0.0.1:50984 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 18:27:51.699981 7825 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213640155572-7825-0/krb5kdc/krb5cc' of type FILE
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:51.700994 7825 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:51.701893 7825 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.65@KRBTEST.COM
I20260808 18:27:51.710629 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:51.702763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51000 (local address 127.7.164.65:46053)
0808 18:27:51.702886 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:51.702890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:51.703009 (+ 119us) server_negotiation.cc:407] Connection header received
0808 18:27:51.703138 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:51.703141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:51.703186 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:51.703266 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:51.703754 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.704318 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.705127 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:51.705420 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:51.705525 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:51.707765 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:51.707769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:51.707772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:51.707798 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:51.709001 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:51.709423 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:51.709427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:51.709429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:51.709515 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:51.709790 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:51.709795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:51.709796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:51.709966 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:51.710130 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:51.710361 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 18:27:51.710465 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:27:51.711755 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 16303
I20260808 18:27:51.724191 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 16493
I20260808 18:27:51.744451 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 16700
I20260808 18:27:51.762600 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 16169
2026-08-08T18:27:51Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2444 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16918](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16918](info): set up 2 sockets
Aug 08 18:27:51 dist-test-slave-jztv krb5kdc[16918](info): commencing operation
krb5kdc: starting...
W20260808 18:27:52.991550 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:27:53 dist-test-slave-jztv krb5kdc[16918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:53Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.126.keytab.
Entry for principal oryx/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.126.keytab.
I20260808 18:27:53.130095 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:35069
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.126.keytab
--principal=oryx/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:35069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:53.252413 16939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:53.252776 16939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:53.252859 16939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:53.256978 16939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:53.257086 16939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:53.257120 16939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:53.257164 16939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:53.257196 16939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:53.262745 16939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:35069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.126.keytab
--principal=oryx/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:35069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:53.263928 16939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:53.265014 16939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:53.274036 16952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:53.274088 16950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:53.274026 16948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:53.274282 16939 server_base.cc:1034] running on GCE node
I20260808 18:27:53.275229 16939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:53.276463 16939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:53.277678 16939 hybrid_clock.cc:648] HybridClock initialized: now 1786213673277665 us; error 33 us; skew 500 ppm
Aug 08 18:27:53 dist-test-slave-jztv krb5kdc[16918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213673, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:53.284116 16939 init.cc:377] Logged in from keytab as oryx/127.7.164.126@KRBTEST.COM (short username oryx)
I20260808 18:27:53.285157 16939 webserver.cc:460] Webserver started at http://127.7.164.126:43371/ using document root <none> and password file <none>
I20260808 18:27:53.285979 16939 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:53.286095 16939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:53.286399 16939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:53.288594 16939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "b0eb5242ae394100a33e0ced51baccb6"
format_stamp: "Formatted at 2026-08-08 18:27:53 on dist-test-slave-jztv"
server_key: "3252ceb27c292d178c67d574322047ff"
server_key_iv: "ac090d463039bf7a24e052946ea5b413"
server_key_version: "encryptionkey@0"
I20260808 18:27:53.289199 16939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "b0eb5242ae394100a33e0ced51baccb6"
format_stamp: "Formatted at 2026-08-08 18:27:53 on dist-test-slave-jztv"
server_key: "3252ceb27c292d178c67d574322047ff"
server_key_iv: "ac090d463039bf7a24e052946ea5b413"
server_key_version: "encryptionkey@0"
I20260808 18:27:53.293468 16939 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 18:27:53.296154 16959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:53.297477 16939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:27:53.297628 16939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "b0eb5242ae394100a33e0ced51baccb6"
format_stamp: "Formatted at 2026-08-08 18:27:53 on dist-test-slave-jztv"
server_key: "3252ceb27c292d178c67d574322047ff"
server_key_iv: "ac090d463039bf7a24e052946ea5b413"
server_key_version: "encryptionkey@0"
I20260808 18:27:53.297747 16939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:53.316016 16939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:53.319350 16939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:53.319589 16939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:53.326943 16939 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:35069
I20260808 18:27:53.327257 17041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:35069 every 8 connection(s)
I20260808 18:27:53.327975 16939 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:53.328806 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 16939
I20260808 18:27:53.328948 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:53.329308 7825 external_mini_cluster.cc:1428] Setting key 1878e4985603073da64dff5e180a6dd5
I20260808 18:27:53.331476 17042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:53.338163 17042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6: Bootstrap starting.
Aug 08 18:27:53 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
I20260808 18:27:53.340502 17042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:53.341234 17042 log.cc:824] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:53.343207 17042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6: No bootstrap required, opened a new log
I20260808 18:27:53.344143 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:53.330812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46054 (local address 127.7.164.126:35069)
0808 18:27:53.331247 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:53.331264 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:53.331303 (+ 39us) server_negotiation.cc:407] Connection header received
0808 18:27:53.332004 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:53.332023 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:53.332364 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:53.332653 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:53.333576 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:53.334679 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:53.335789 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:53.336253 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:53.336374 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:53.339263 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:53.339268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:53.339282 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:53.339311 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:53.341143 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:53.341664 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:53.341669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:53.341674 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:53.341793 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:53.342094 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:53.342097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:53.342099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:53.342568 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:53.342776 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:53.343089 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 18:27:53.343428 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":145,"threads_started":1}
I20260808 18:27:53.346937 17042 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } }
I20260808 18:27:53.347237 17042 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:53.347316 17042 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0eb5242ae394100a33e0ced51baccb6, State: Initialized, Role: FOLLOWER
I20260808 18:27:53.347728 17042 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } }
I20260808 18:27:53.347873 17042 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:53.347952 17042 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:53.348057 17042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:53.348939 17042 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } }
I20260808 18:27:53.349280 17042 leader_election.cc:302] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0eb5242ae394100a33e0ced51baccb6; no voters:
I20260808 18:27:53.349597 17042 leader_election.cc:288] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:53.349745 17050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:53.349987 17050 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [term 1 LEADER]: Becoming Leader. State: Replica: b0eb5242ae394100a33e0ced51baccb6, State: Running, Role: LEADER
I20260808 18:27:53.350399 17050 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } }
I20260808 18:27:53.350853 17042 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:53.352118 17053 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0eb5242ae394100a33e0ced51baccb6. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } }
I20260808 18:27:53.352165 17051 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } }
I20260808 18:27:53.352360 17053 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:53.352378 17051 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:53.353030 17062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:53.356114 17062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:53.361492 17062 catalog_manager.cc:1347] Generated new cluster ID: 772ce27b4ff4457e9e891cd52b827ba7
I20260808 18:27:53.361564 17062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:53.386256 17062 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:53.387225 17062 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:53.391248 17062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6: Generated new TSK 0
I20260808 18:27:53.391916 17062 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.65.keytab.
Entry for principal oryx/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.65.keytab.
I20260808 18:27:53.429867 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35069
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.65.keytab
--principal=oryx/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:53.545493 17076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:53.545758 17076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:53.545801 17076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:53.549908 17076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:53.549969 17076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:53.550064 17076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:27:53.555660 17076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.65.keytab
--principal=oryx/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:53.556707 17076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:53.557740 17076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:53.567396 17083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:53.567390 17087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:53.567390 17082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:53.567441 17076 server_base.cc:1034] running on GCE node
I20260808 18:27:53.567986 17076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:53.568606 17076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:53.569783 17076 hybrid_clock.cc:648] HybridClock initialized: now 1786213673569750 us; error 45 us; skew 500 ppm
Aug 08 18:27:53 dist-test-slave-jztv krb5kdc[16918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213673, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:53.572428 17076 init.cc:377] Logged in from keytab as oryx/127.7.164.65@KRBTEST.COM (short username oryx)
I20260808 18:27:53.573241 17076 webserver.cc:460] Webserver started at http://127.7.164.65:33261/ using document root <none> and password file <none>
I20260808 18:27:53.573786 17076 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:53.573874 17076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:53.574111 17076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:53.576110 17076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "b328e02ab41241ec9edc85a11d4da0fe"
format_stamp: "Formatted at 2026-08-08 18:27:53 on dist-test-slave-jztv"
server_key: "626cfd7157ca67206342761087f9e907"
server_key_iv: "a74ba4a20fec21cc945e5cef6019658f"
server_key_version: "encryptionkey@0"
I20260808 18:27:53.576663 17076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "b328e02ab41241ec9edc85a11d4da0fe"
format_stamp: "Formatted at 2026-08-08 18:27:53 on dist-test-slave-jztv"
server_key: "626cfd7157ca67206342761087f9e907"
server_key_iv: "a74ba4a20fec21cc945e5cef6019658f"
server_key_version: "encryptionkey@0"
I20260808 18:27:53.580268 17076 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:27:53.582679 17096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:53.583843 17076 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:53.583976 17076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "b328e02ab41241ec9edc85a11d4da0fe"
format_stamp: "Formatted at 2026-08-08 18:27:53 on dist-test-slave-jztv"
server_key: "626cfd7157ca67206342761087f9e907"
server_key_iv: "a74ba4a20fec21cc945e5cef6019658f"
server_key_version: "encryptionkey@0"
I20260808 18:27:53.584093 17076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:53.594769 17076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:53.597561 17076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:53.597759 17076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:53.598404 17076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:53.599262 17076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:53.599344 17076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:53.599432 17076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:53.599478 17076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:53.608999 17076 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:33189
I20260808 18:27:53.609018 17253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:33189 every 8 connection(s)
I20260808 18:27:53.609946 17076 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:27:53.616680 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 17076
I20260808 18:27:53.616827 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:27:53.617151 7825 external_mini_cluster.cc:1428] Setting key 4846d75b7de04d0a49685c3aadd3c32d
Aug 08 18:27:53 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213673, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
I20260808 18:27:53.625268 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:53.611904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:43475 (local address 127.7.164.126:35069)
0808 18:27:53.612121 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:53.612128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:53.612991 (+ 863us) server_negotiation.cc:407] Connection header received
0808 18:27:53.614395 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:53.614399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:53.614462 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:53.614577 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:53.615809 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:53.616356 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:53.617874 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:53.618164 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:53.618349 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:53.621268 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:53.621274 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:53.621278 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:53.621317 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:53.622954 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:53.623614 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:53.623617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:53.623618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:53.623695 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:53.624114 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:53.624117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:53.624118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:53.624297 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:53.624413 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:53.624942 (+ 529us) server_negotiation.cc:299] Negotiation successful
0808 18:27:53.625090 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:27:53.626106 17262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:53.612281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.65:43475)
0808 18:27:53.612827 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:53.612887 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:53.614138 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:53.614760 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:53.614766 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:53.615157 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:53.615652 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:53.615666 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:53.616554 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:53.616557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:53.617745 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:53.617758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:53.618414 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:53.618419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:53.618661 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:53.619354 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:53.619371 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:53.621089 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:53.623079 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:53.623088 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:53.623107 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:53.623490 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:53.623783 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:53.623789 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:53.623794 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:53.624016 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:53.624417 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:53.624426 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:53.624695 (+ 269us) client_negotiation.cc:769] Sending connection context
0808 18:27:53.624897 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 18:27:53.625240 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":146,"threads_started":1}
I20260808 18:27:53.627583 17259 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35069
I20260808 18:27:53.627846 17259 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:53.628371 17259 heartbeater.cc:511] Master 127.7.164.126:35069 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:27:53.630020 16986 ts_manager.cc:194] Registered new tserver with Master: b328e02ab41241ec9edc85a11d4da0fe (127.7.164.65:33189)
I20260808 18:27:53.631219 16986 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:43475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.164.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.66.keytab.
Entry for principal oryx/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.66.keytab.
I20260808 18:27:53.647575 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35069
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.66.keytab
--principal=oryx/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:53.762609 17267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:53.762917 17267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:53.762989 17267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:53.767127 17267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:53.767208 17267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:53.767299 17267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:27:53.772552 17267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.66.keytab
--principal=oryx/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:53.773700 17267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:53.774812 17267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:53.784859 17274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:53.784879 17277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:53.784874 17273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:53.785107 17267 server_base.cc:1034] running on GCE node
I20260808 18:27:53.785575 17267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:53.786250 17267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:53.787433 17267 hybrid_clock.cc:648] HybridClock initialized: now 1786213673787402 us; error 41 us; skew 500 ppm
Aug 08 18:27:53 dist-test-slave-jztv krb5kdc[16918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213673, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:53.790345 17267 init.cc:377] Logged in from keytab as oryx/127.7.164.66@KRBTEST.COM (short username oryx)
I20260808 18:27:53.791429 17267 webserver.cc:460] Webserver started at http://127.7.164.66:39091/ using document root <none> and password file <none>
I20260808 18:27:53.792019 17267 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:53.792110 17267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:53.792394 17267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:53.794925 17267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "49a0cb2e3dad4a548936000ec3d80547"
format_stamp: "Formatted at 2026-08-08 18:27:53 on dist-test-slave-jztv"
server_key: "7ed992c131e832abe1343fe6828babe3"
server_key_iv: "f65fc7a123b920ae0369ad69e551631d"
server_key_version: "encryptionkey@0"
I20260808 18:27:53.795617 17267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "49a0cb2e3dad4a548936000ec3d80547"
format_stamp: "Formatted at 2026-08-08 18:27:53 on dist-test-slave-jztv"
server_key: "7ed992c131e832abe1343fe6828babe3"
server_key_iv: "f65fc7a123b920ae0369ad69e551631d"
server_key_version: "encryptionkey@0"
I20260808 18:27:53.799535 17267 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:27:53.802276 17286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:53.803570 17267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:27:53.803821 17267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "49a0cb2e3dad4a548936000ec3d80547"
format_stamp: "Formatted at 2026-08-08 18:27:53 on dist-test-slave-jztv"
server_key: "7ed992c131e832abe1343fe6828babe3"
server_key_iv: "f65fc7a123b920ae0369ad69e551631d"
server_key_version: "encryptionkey@0"
I20260808 18:27:53.803946 17267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:53.823751 17267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:53.827346 17267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:53.827581 17267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:53.828219 17267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:53.829113 17267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:53.829182 17267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:53.829255 17267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:53.829295 17267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:53.838497 17267 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:37629
I20260808 18:27:53.838565 17454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:37629 every 8 connection(s)
I20260808 18:27:53.839506 17267 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:27:53.845017 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 17267
I20260808 18:27:53.845116 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:27:53.845413 7825 external_mini_cluster.cc:1428] Setting key 54f3b8eb1bc21881cb1e15cca8a181c9
Aug 08 18:27:53 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213673, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
I20260808 18:27:53.854627 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:53.841415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:42075 (local address 127.7.164.126:35069)
0808 18:27:53.841564 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:53.841568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:53.842540 (+ 972us) server_negotiation.cc:407] Connection header received
0808 18:27:53.844118 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:53.844120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:53.844171 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:53.844262 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:53.845528 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:53.846166 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:53.847487 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:53.847711 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:53.847822 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:53.850371 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:53.850374 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:53.850376 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:53.850402 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:53.852024 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:53.852717 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:53.852721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:53.852722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:53.852802 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:53.853265 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:53.853268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:53.853270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:53.853513 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:53.853606 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:53.854357 (+ 751us) server_negotiation.cc:299] Negotiation successful
0808 18:27:53.854495 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:27:53.855840 17459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:53.841834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.66:42075)
0808 18:27:53.842341 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:53.842395 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:53.843893 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:53.844430 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:53.844437 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:53.844807 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:53.845373 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:53.845391 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:53.846352 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:53.846355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:53.847353 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:53.847364 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:53.847864 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:53.847868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:53.848040 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:53.848654 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:53.848671 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:53.850183 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:53.852147 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:53.852157 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:53.852175 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:53.852597 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:53.852911 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:53.852918 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:53.852922 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:53.853131 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:53.853638 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:53.853649 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:53.854019 (+ 370us) client_negotiation.cc:769] Sending connection context
0808 18:27:53.854280 (+ 261us) client_negotiation.cc:240] Negotiation successful
0808 18:27:53.854670 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":148,"threads_started":1}
WARNING: no policy specified for oryx/127.7.164.67@KRBTEST.COM; defaulting to no policy
I20260808 18:27:53.857177 17455 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35069
I20260808 18:27:53.857407 17455 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:53.857975 17455 heartbeater.cc:511] Master 127.7.164.126:35069 requested a full tablet report, sending...
I20260808 18:27:53.859112 16986 ts_manager.cc:194] Registered new tserver with Master: 49a0cb2e3dad4a548936000ec3d80547 (127.7.164.66:37629)
I20260808 18:27:53.859701 16986 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:42075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.164.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.67.keytab.
Entry for principal oryx/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.67.keytab.
I20260808 18:27:53.877025 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35069
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.67.keytab
--principal=oryx/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:53.989363 17462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:53.989645 17462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:53.989687 17462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:53.993696 17462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:53.993760 17462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:53.993820 17462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:27:53.999116 17462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.67.keytab
--principal=oryx/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:54.000161 17462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:54.001129 17462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:54.010708 17471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:54.010730 17469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:54.010730 17468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:54.010782 17462 server_base.cc:1034] running on GCE node
I20260808 18:27:54.011437 17462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:54.012076 17462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:54.013235 17462 hybrid_clock.cc:648] HybridClock initialized: now 1786213674013205 us; error 44 us; skew 500 ppm
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:54.015947 17462 init.cc:377] Logged in from keytab as oryx/127.7.164.67@KRBTEST.COM (short username oryx)
I20260808 18:27:54.016810 17462 webserver.cc:460] Webserver started at http://127.7.164.67:43183/ using document root <none> and password file <none>
I20260808 18:27:54.017377 17462 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:54.017468 17462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:54.017701 17462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:54.019826 17462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "d3db78488e4f4f7489023ebb7002a030"
format_stamp: "Formatted at 2026-08-08 18:27:54 on dist-test-slave-jztv"
server_key: "321e8575687fff667dbdd1665ce3cc43"
server_key_iv: "b625eccd618132bef0cf58688ba629d4"
server_key_version: "encryptionkey@0"
I20260808 18:27:54.020406 17462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "d3db78488e4f4f7489023ebb7002a030"
format_stamp: "Formatted at 2026-08-08 18:27:54 on dist-test-slave-jztv"
server_key: "321e8575687fff667dbdd1665ce3cc43"
server_key_iv: "b625eccd618132bef0cf58688ba629d4"
server_key_version: "encryptionkey@0"
I20260808 18:27:54.024125 17462 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:27:54.026454 17479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:54.027587 17462 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:54.027717 17462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "d3db78488e4f4f7489023ebb7002a030"
format_stamp: "Formatted at 2026-08-08 18:27:54 on dist-test-slave-jztv"
server_key: "321e8575687fff667dbdd1665ce3cc43"
server_key_iv: "b625eccd618132bef0cf58688ba629d4"
server_key_version: "encryptionkey@0"
I20260808 18:27:54.027822 17462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:54.059662 17462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:54.062489 17462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:54.062695 17462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:54.063277 17462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:54.064143 17462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:54.064210 17462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:54.064280 17462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:54.064320 17462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:54.073351 17462 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:42685
I20260808 18:27:54.073376 17643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:42685 every 8 connection(s)
I20260808 18:27:54.074337 17462 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
I20260808 18:27:54.084488 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 17462
I20260808 18:27:54.084630 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:27:54.085026 7825 external_mini_cluster.cc:1428] Setting key 1834af5f4255d54c5797fb4c76c9e669
I20260808 18:27:54.088304 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.076107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:48703 (local address 127.7.164.126:35069)
0808 18:27:54.076244 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.076247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.077074 (+ 827us) server_negotiation.cc:407] Connection header received
0808 18:27:54.078459 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.078462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.078521 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.078624 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.079774 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.080300 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.081409 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.081624 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.081733 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.084146 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.084149 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.084152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.084200 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.085892 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.086641 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.086647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.086650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.086737 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.087099 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.087106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.087107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.087370 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.087481 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.088003 (+ 522us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.088134 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:27:54.089087 17648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.076497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.67:48703)
0808 18:27:54.076928 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.076970 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.078201 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.078797 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.078804 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.079156 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.079631 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.079644 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.080423 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.080426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.081279 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.081288 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.081775 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.081779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.081948 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.082533 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.082560 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.083984 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.086036 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.086043 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.086061 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.086524 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.086838 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.086841 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.086843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.086986 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.087481 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.087489 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.087763 (+ 274us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.087943 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.088268 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":112,"threads_started":1}
I20260808 18:27:54.090147 17646 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35069
I20260808 18:27:54.090405 17646 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:54.090860 17646 heartbeater.cc:511] Master 127.7.164.126:35069 requested a full tablet report, sending...
I20260808 18:27:54.091789 16986 ts_manager.cc:194] Registered new tserver with Master: d3db78488e4f4f7489023ebb7002a030 (127.7.164.67:42685)
I20260808 18:27:54.092342 16986 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:48703
I20260808 18:27:54.098852 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.164.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.164.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.164.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.125.keytab.
Entry for principal oryx/127.7.164.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.125.keytab.
I20260808 18:27:54.128736 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.125:43073
--webserver_interface=127.7.164.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.125.keytab
--principal=oryx/127.7.164.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:35069,127.7.164.125:43073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:54.242101 17652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:54.242415 17652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:54.242460 17652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:54.246402 17652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:54.246450 17652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:54.246470 17652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:54.246486 17652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:54.246501 17652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:54.251742 17652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:35069,127.7.164.125:43073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.125.keytab
--principal=oryx/127.7.164.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.125:43073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.164.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
W20260808 18:27:54.251969 17652 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.164.126:35069,127.7.164.125:43073'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 18:27:54.252805 17652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:54.253785 17652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:54.262319 17660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:54.262321 17658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:54.262321 17662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:54.262503 17652 server_base.cc:1034] running on GCE node
I20260808 18:27:54.263065 17652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:54.264114 17652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:54.265327 17652 hybrid_clock.cc:648] HybridClock initialized: now 1786213674265300 us; error 42 us; skew 500 ppm
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:54.267964 17652 init.cc:377] Logged in from keytab as oryx/127.7.164.125@KRBTEST.COM (short username oryx)
I20260808 18:27:54.268858 17652 webserver.cc:460] Webserver started at http://127.7.164.125:41249/ using document root <none> and password file <none>
I20260808 18:27:54.269407 17652 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:54.269497 17652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:54.269726 17652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:54.271853 17652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/data/instance:
uuid: "49041fa054ed40ca9242a05b3d0eaddb"
format_stamp: "Formatted at 2026-08-08 18:27:54 on dist-test-slave-jztv"
server_key: "c7b4d1c4a39de1063732545cc1585fe5"
server_key_iv: "98a9a736bb826e81f0cb41e2b1081445"
server_key_version: "encryptionkey@0"
I20260808 18:27:54.272419 17652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/wal/instance:
uuid: "49041fa054ed40ca9242a05b3d0eaddb"
format_stamp: "Formatted at 2026-08-08 18:27:54 on dist-test-slave-jztv"
server_key: "c7b4d1c4a39de1063732545cc1585fe5"
server_key_iv: "98a9a736bb826e81f0cb41e2b1081445"
server_key_version: "encryptionkey@0"
I20260808 18:27:54.276250 17652 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:27:54.278714 17669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:54.280440 17652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:27:54.280599 17652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/wal
uuid: "49041fa054ed40ca9242a05b3d0eaddb"
format_stamp: "Formatted at 2026-08-08 18:27:54 on dist-test-slave-jztv"
server_key: "c7b4d1c4a39de1063732545cc1585fe5"
server_key_iv: "98a9a736bb826e81f0cb41e2b1081445"
server_key_version: "encryptionkey@0"
I20260808 18:27:54.280741 17652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:54.302644 17652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:54.305500 17652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:54.305660 17652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:54.312866 17652 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.125:43073
I20260808 18:27:54.312887 17746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.125:43073 every 8 connection(s)
I20260808 18:27:54.313822 17652 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/data/info.pb
I20260808 18:27:54.316229 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 17652
I20260808 18:27:54.316797 17747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:54.321141 17747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.164.125@KRBTEST.COM
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.125@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
I20260808 18:27:54.331797 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.317895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.7.164.125:43073)
0808 18:27:54.318358 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.318375 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.318410 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:27:54.319059 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.319074 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.319399 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.319714 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.320512 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.321520 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.322732 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.323418 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.323573 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.326171 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.326176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.326185 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.326252 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.328096 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.328637 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.328643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.328649 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.328775 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.329105 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.329108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.329110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.329616 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.329848 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.330348 (+ 500us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.330697 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":133,"threads_started":1}
I20260808 18:27:54.332556 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.323142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46070 (local address 127.7.164.126:35069)
0808 18:27:54.323295 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.323300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.323598 (+ 298us) server_negotiation.cc:407] Connection header received
0808 18:27:54.323811 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.323813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.323859 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.323951 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.324602 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.325132 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.326076 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.326382 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.326491 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.329020 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.329024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.329027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.329054 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.330707 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.331209 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.331212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.331214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.331294 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.331584 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.331587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.331589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.331837 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.331926 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.332299 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.332412 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:27:54.333093 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.323074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.0.0.1:46070)
0808 18:27:54.323454 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.323520 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.323698 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.324000 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.324005 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.324351 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.324472 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.324478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.325265 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.325268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.325959 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.325968 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.326494 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.326498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.326651 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.327197 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.327212 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.328874 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.330818 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.330825 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.330830 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.331100 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.331386 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.331389 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.331391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.331487 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.331979 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.331984 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.332094 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.332746 (+ 652us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.332931 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":145,"threads_started":1}
I20260808 18:27:54.334375 17747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.125@KRBTEST.COM for oryx/127.7.164.125@KRBTEST.COM
I20260808 18:27:54.342782 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.334855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:43073 (local address 127.0.0.1:53512)
0808 18:27:54.334991 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.335004 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.335067 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.335270 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.335273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.335430 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.335978 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.335986 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.336666 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.336669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.337478 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.337488 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.337900 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.337903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.337992 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.338430 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.338443 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.339722 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.341347 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.341351 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.341353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.341540 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.341808 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.341810 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.341812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.341891 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.342285 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.342289 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.342370 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.342476 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.342613 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 18:27:54.342785 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.334921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53512 (local address 127.7.164.125:43073)
0808 18:27:54.335022 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.335026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.335042 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:27:54.335142 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.335144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.335177 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.335240 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.336083 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.336551 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.337595 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.337804 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.337879 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.339853 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.339857 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.339859 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.339885 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.341222 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.341633 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.341639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.341640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.341714 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.341977 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.341982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.341984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.342150 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.342272 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.342522 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.342672 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":36}
I20260808 18:27:54.345543 17747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: Bootstrap starting.
I20260808 18:27:54.347832 17747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:54.348505 17747 log.cc:824] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:54.350329 17747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: No bootstrap required, opened a new log
I20260808 18:27:54.353514 17747 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } }
I20260808 18:27:54.353739 17747 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:54.353785 17747 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49041fa054ed40ca9242a05b3d0eaddb, State: Initialized, Role: FOLLOWER
I20260808 18:27:54.354203 17747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } }
I20260808 18:27:54.354696 17755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } } }
I20260808 18:27:54.354823 17755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:54.355472 17747 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: configured and running, proceeding with master startup.
W20260808 18:27:54.360090 17769 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:27:54.360174 17769 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:27:54.363020 17652 master_runner.cc:428] Detected that this master 49041fa054ed40ca9242a05b3d0eaddb is joining an existing cluster
I20260808 18:27:54.363073 17652 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.164.125:43073
I20260808 18:27:54.373575 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.365224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46076 (local address 127.7.164.126:35069)
0808 18:27:54.365353 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.365358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.365623 (+ 265us) server_negotiation.cc:407] Connection header received
0808 18:27:54.365801 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.365804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.365856 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.365940 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.366599 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.367147 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.368276 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.368659 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.368796 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.370275 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.370280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.370285 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.370312 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.371850 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.372331 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.372335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.372337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.372433 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.372707 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.372710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.372712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.372903 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.373012 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.373291 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.373407 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:27:54.373593 17779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.365265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:43073 (local address 127.0.0.1:53524)
0808 18:27:54.365690 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.365703 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.365834 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.366130 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.366132 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.366397 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.366493 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.366499 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.367584 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.367587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.368367 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.368374 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.368780 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.368783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.368892 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.369408 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.369428 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.369885 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.371822 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.371828 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.371830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.372159 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.372462 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.372465 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.372467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.372557 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.372984 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.372988 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.373080 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.373205 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.373363 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":3,"spinlock_wait_cycles":4480,"thread_start_us":98,"threads_started":1}
I20260808 18:27:54.373600 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.365611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53524 (local address 127.7.164.125:43073)
0808 18:27:54.365713 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.365717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.365773 (+ 56us) server_negotiation.cc:407] Connection header received
0808 18:27:54.365937 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.365940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.365978 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.366047 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.366651 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.367457 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.368467 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.368670 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.368772 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.370010 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.370014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.370017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.370039 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.371698 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.372262 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.372268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.372271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.372363 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.372654 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.372658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.372660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.372874 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.372975 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.373265 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.373422 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":46}
I20260808 18:27:54.374048 17777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.365191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.0.0.1:46076)
0808 18:27:54.365523 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.365555 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.365699 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.365970 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.365973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.366322 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.366468 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.366477 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.367253 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.367258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.368146 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.368156 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.368954 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.368959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.369053 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.369576 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.369593 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.370102 (+ 509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.371951 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.371955 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.371958 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.372187 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.372514 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.372516 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.372518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.372588 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.373051 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.373054 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.373117 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.373799 (+ 682us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.373938 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213673, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.126@KRBTEST.COM for oryx/127.7.164.125@KRBTEST.COM
I20260808 18:27:54.389220 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.378669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:43073 (local address 127.0.0.1:53526)
0808 18:27:54.379034 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.379082 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.379212 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.379539 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.379544 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.379889 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.379996 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.380002 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.380983 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.380985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.381618 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.381627 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.382194 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.382196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.382327 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.382881 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.382895 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.384697 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.386854 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.386865 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.386873 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.387357 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.387743 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.387746 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.387749 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.387852 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.388471 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.388479 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.388623 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.388786 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.389008 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":102,"threads_started":1}
I20260808 18:27:54.389216 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.378716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53526 (local address 127.7.164.125:43073)
0808 18:27:54.378832 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.378837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.379157 (+ 320us) server_negotiation.cc:407] Connection header received
0808 18:27:54.379366 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.379370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.379424 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.379505 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.380107 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.380860 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.381726 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.382059 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.382168 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.384880 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.384884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.384887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.384910 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.386733 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.387458 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.387464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.387469 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.387569 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.387967 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.387971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.387973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.388319 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.388412 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.388892 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.389028 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:27:54.390925 16986 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.164.125:43073: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb"
member_type: NON_VOTER
last_known_addr {
host: "127.7.164.125"
port: 43073
}
attrs {
promote: true
}
}
dest_uuid: "b0eb5242ae394100a33e0ced51baccb6"
cas_config_opid_index: -1
I20260808 18:27:54.391774 16986 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } }
I20260808 18:27:54.393966 17789 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 49041fa054ed40ca9242a05b3d0eaddb (127.7.164.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } }
I20260808 18:27:54.394567 17709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:54.396032 17709 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 FOLLOWER]: Refusing update from remote peer b0eb5242ae394100a33e0ced51baccb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 18:27:54.396387 17789 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.164.125:43073
WARNING: no policy specified for oryx/127.7.164.124@KRBTEST.COM; defaulting to no policy
I20260808 18:27:54.396958 17652 master.cc:537] Master@127.7.164.125:43073 shutting down...
I20260808 18:27:54.397006 17788 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } }
I20260808 18:27:54.397135 17788 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:54.397442 17791 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.001s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.164.124@KRBTEST.COM" created.
W20260808 18:27:54.402838 16963 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 18:27:54.403820 16963 consensus_peers.cc:487] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 -> Peer 49041fa054ed40ca9242a05b3d0eaddb (127.7.164.125:43073): Couldn't send request to peer 49041fa054ed40ca9242a05b3d0eaddb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 18:27:54.403996 17652 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:54.404289 17652 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:54.404413 17652 tablet_replica.cc:330] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.164.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.124.keytab.
Entry for principal oryx/127.7.164.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.124.keytab.
I20260808 18:27:54.419361 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.124:38159
--webserver_interface=127.7.164.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.124.keytab
--principal=oryx/127.7.164.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:35069,127.7.164.125:43073,127.7.164.124:38159
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:27:54.420380 17652 master.cc:552] Master@127.7.164.125:43073 shutdown complete.
I20260808 18:27:54.420454 17652 master_runner.cc:305] Clearing existing system tablet
I20260808 18:27:54.420701 17652 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:54.420773 17652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:54.422333 17652 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 18:27:54.423374 17803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:54.423630 17652 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:54.423808 17652 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:54.423854 17652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/wal
uuid: "49041fa054ed40ca9242a05b3d0eaddb"
format_stamp: "Formatted at 2026-08-08 18:27:54 on dist-test-slave-jztv"
server_key: "c7b4d1c4a39de1063732545cc1585fe5"
server_key_iv: "98a9a736bb826e81f0cb41e2b1081445"
server_key_version: "encryptionkey@0"
I20260808 18:27:54.427423 17652 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:54.430500 17652 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 18:27:54.430625 17652 log.cc:1196] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 18:27:54.430934 17652 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: Deleting consensus metadata
I20260808 18:27:54.431137 17652 master_runner.cc:315] Copying system tablet from 127.7.164.126:35069
I20260808 18:27:54.432765 17652 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: tablet copy: Beginning tablet copy session from remote peer at address 127.7.164.126:35069
I20260808 18:27:54.442135 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.433583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46078 (local address 127.7.164.126:35069)
0808 18:27:54.433716 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.433721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.433901 (+ 180us) server_negotiation.cc:407] Connection header received
0808 18:27:54.434044 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.434048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.434099 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.434197 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.434791 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.435631 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.436804 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.437149 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.437249 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.438608 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.438612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.438615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.438643 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.440392 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.440887 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.440891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.440893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.440980 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.441269 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.441273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.441274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.441478 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.441574 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.441863 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.441976 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:27:54.442150 17813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.433474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.0.0.1:46078)
0808 18:27:54.433789 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.433805 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.433950 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.434267 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.434270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.434519 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.434665 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.434674 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.435754 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.435758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.436679 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.436690 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.437267 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.437271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.437365 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.437967 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.437982 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.438453 (+ 471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.440509 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.440513 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.440516 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.440778 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.441075 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.441078 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.441080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.441176 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.441586 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.441591 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.441692 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.441797 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.441935 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":86,"threads_started":1}
I20260808 18:27:54.443051 17018 tablet_copy_service.cc:140] P b0eb5242ae394100a33e0ced51baccb6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 49041fa054ed40ca9242a05b3d0eaddb ({username='oryx', principal='oryx/127.7.164.125@KRBTEST.COM'} at 127.0.0.1:46078)
I20260808 18:27:54.443238 17018 tablet_copy_service.cc:161] P b0eb5242ae394100a33e0ced51baccb6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 49041fa054ed40ca9242a05b3d0eaddb at {username='oryx', principal='oryx/127.7.164.125@KRBTEST.COM'} at 127.0.0.1:46078: session id = 49041fa054ed40ca9242a05b3d0eaddb-00000000000000000000000000000000
I20260808 18:27:54.445266 17018 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 18:27:54.447187 17652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 18:27:54.449604 17652 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: tablet copy: Starting download of 0 data blocks...
I20260808 18:27:54.449805 17652 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: tablet copy: Starting download of 1 WAL segments...
I20260808 18:27:54.451702 17652 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 18:27:54.453575 17018 tablet_copy_service.cc:342] P b0eb5242ae394100a33e0ced51baccb6: Request end of tablet copy session 49041fa054ed40ca9242a05b3d0eaddb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.164.125@KRBTEST.COM'} at 127.0.0.1:46078
I20260808 18:27:54.453696 17018 tablet_copy_service.cc:433] P b0eb5242ae394100a33e0ced51baccb6: ending tablet copy session 49041fa054ed40ca9242a05b3d0eaddb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 49041fa054ed40ca9242a05b3d0eaddb
I20260808 18:27:54.461966 17652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:54.464300 17815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:54.464502 17814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:54.464743 17652 server_base.cc:1034] running on GCE node
W20260808 18:27:54.466667 17819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:54.466890 17652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:54.467208 17652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:54.468364 17652 hybrid_clock.cc:648] HybridClock initialized: now 1786213674468340 us; error 41 us; skew 500 ppm
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:54.469998 17652 init.cc:377] Logged in from keytab as oryx/127.7.164.125@KRBTEST.COM (short username oryx)
I20260808 18:27:54.470520 17652 webserver.cc:460] Webserver started at http://127.7.164.125:46805/ using document root <none> and password file <none>
I20260808 18:27:54.470721 17652 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:54.470768 17652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:54.472220 17652 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:54.473598 17830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:54.473874 17652 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:54.473955 17652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/wal
uuid: "49041fa054ed40ca9242a05b3d0eaddb"
format_stamp: "Formatted at 2026-08-08 18:27:54 on dist-test-slave-jztv"
server_key: "c7b4d1c4a39de1063732545cc1585fe5"
server_key_iv: "98a9a736bb826e81f0cb41e2b1081445"
server_key_version: "encryptionkey@0"
I20260808 18:27:54.474043 17652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:54.496944 17652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:54.499310 17652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:54.504819 17652 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.125:43073
I20260808 18:27:54.504848 17905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.125:43073 every 8 connection(s)
I20260808 18:27:54.505458 17652 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-1/data/info.pb
I20260808 18:27:54.507609 17906 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:27:54.508744 17906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: Bootstrap starting.
I20260808 18:27:54.519788 17906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:27:54.520117 17906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: Bootstrap complete.
I20260808 18:27:54.520684 17906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } }
I20260808 18:27:54.520820 17906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 49041fa054ed40ca9242a05b3d0eaddb, State: Initialized, Role: LEARNER
I20260808 18:27:54.520913 17906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } }
I20260808 18:27:54.521288 17909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } }
I20260808 18:27:54.521384 17906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:54.521510 17909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: This master's current role is: LEARNER
I20260808 18:27:54.524829 17930 catalog_manager.cc:1259] Loaded cluster ID: 772ce27b4ff4457e9e891cd52b827ba7
I20260808 18:27:54.524952 17930 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: loading cluster ID for follower catalog manager: success
I20260808 18:27:54.527127 17930 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: acquiring CA information for follower catalog manager: success
I20260808 18:27:54.528520 17930 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 18:27:54.553911 17799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:54.554160 17799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:54.554256 17799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:54.558111 17799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:54.558166 17799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:54.558185 17799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:54.558202 17799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:54.558246 17799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:54.563491 17799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:35069,127.7.164.125:43073,127.7.164.124:38159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.124.keytab
--principal=oryx/127.7.164.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.124:38159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.164.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:54.564590 17799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:54.565737 17799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:27:54.574321 17799 server_base.cc:1034] running on GCE node
W20260808 18:27:54.574280 17937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:54.574302 17938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:54.574280 17943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:54.575214 17799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:54.576396 17799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:54.577584 17799 hybrid_clock.cc:648] HybridClock initialized: now 1786213674577547 us; error 55 us; skew 500 ppm
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:54.580750 17799 init.cc:377] Logged in from keytab as oryx/127.7.164.124@KRBTEST.COM (short username oryx)
I20260808 18:27:54.581696 17799 webserver.cc:460] Webserver started at http://127.7.164.124:45885/ using document root <none> and password file <none>
I20260808 18:27:54.582342 17799 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:54.582446 17799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:54.582692 17799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:54.584798 17799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/data/instance:
uuid: "87f431983eab4f48bc40603dca5208f6"
format_stamp: "Formatted at 2026-08-08 18:27:54 on dist-test-slave-jztv"
server_key: "def7857055d48d05ae72e1644886ca42"
server_key_iv: "0f15943a77fb0dc9eb8d54da04e3ea9b"
server_key_version: "encryptionkey@0"
I20260808 18:27:54.585389 17799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/wal/instance:
uuid: "87f431983eab4f48bc40603dca5208f6"
format_stamp: "Formatted at 2026-08-08 18:27:54 on dist-test-slave-jztv"
server_key: "def7857055d48d05ae72e1644886ca42"
server_key_iv: "0f15943a77fb0dc9eb8d54da04e3ea9b"
server_key_version: "encryptionkey@0"
I20260808 18:27:54.589249 17799 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 18:27:54.591801 17952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:54.593000 17799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:27:54.593138 17799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/wal
uuid: "87f431983eab4f48bc40603dca5208f6"
format_stamp: "Formatted at 2026-08-08 18:27:54 on dist-test-slave-jztv"
server_key: "def7857055d48d05ae72e1644886ca42"
server_key_iv: "0f15943a77fb0dc9eb8d54da04e3ea9b"
server_key_version: "encryptionkey@0"
I20260808 18:27:54.593247 17799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:54.620383 17799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:54.623615 17799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:54.623812 17799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:54.631206 17799 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.124:38159
I20260808 18:27:54.631533 18034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.124:38159 every 8 connection(s)
I20260808 18:27:54.632292 17799 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/data/info.pb
I20260808 18:27:54.633661 17259 heartbeater.cc:503] Master 127.7.164.126:35069 was elected leader, sending a full tablet report...
I20260808 18:27:54.635437 18035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:54.637822 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 17799
I20260808 18:27:54.639745 18035 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.124@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.164.124@KRBTEST.COM
I20260808 18:27:54.651847 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.641576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46088 (local address 127.7.164.126:35069)
0808 18:27:54.641701 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.641705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.642062 (+ 357us) server_negotiation.cc:407] Connection header received
0808 18:27:54.642298 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.642301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.642352 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.642424 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.643274 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.643796 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.644818 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.645029 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.645134 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.647711 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.647714 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.647716 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.647742 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.649347 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.649996 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.649998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.650000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.650084 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.650451 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.650456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.650458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.650792 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.650899 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.651448 (+ 549us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.651626 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:27:54.652563 18038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.641432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.0.0.1:46088)
0808 18:27:54.641930 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.641977 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.642155 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.642489 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.642494 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.642920 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.643132 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.643146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.643917 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.643919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.644704 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.644712 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.645167 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.645171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.645323 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.645942 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.645958 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.647511 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.649460 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.649467 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.649477 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.649904 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.650161 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.650163 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.650164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.650351 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.650893 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.650900 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.651174 (+ 274us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.651360 (+ 186us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.651708 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":36,"spinlock_wait_cycles":896,"thread_start_us":136,"threads_started":1}
I20260808 18:27:54.652559 18037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.639561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50878 (local address 127.7.164.124:38159)
0808 18:27:54.640021 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.640036 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.640071 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:27:54.640756 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.640774 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.641109 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.641451 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.642477 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.643726 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.644572 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.645031 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.645197 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.648129 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.648134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.648144 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.648165 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.649904 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.650575 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.650579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.650580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.650647 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.650977 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.650980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.650981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.651239 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.651449 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.651707 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.651849 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":323,"thread_start_us":151,"threads_started":1}
I20260808 18:27:54.653990 18035 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.124@KRBTEST.COM for oryx/127.7.164.125@KRBTEST.COM
I20260808 18:27:54.662312 18038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.654545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:43073 (local address 127.0.0.1:53534)
0808 18:27:54.654694 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.654707 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.654775 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.655145 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.655147 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.655271 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.655365 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.655370 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.656214 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.656217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.656733 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.656740 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.657163 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.657167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.657250 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.657692 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.657703 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.658998 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.660816 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.660821 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.660823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.661023 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.661311 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.661335 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.661338 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.661457 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.661850 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.661852 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.661931 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.662014 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.662117 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260808 18:27:54.662370 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.654660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53534 (local address 127.7.164.125:43073)
0808 18:27:54.654908 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.654912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.654925 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:27:54.654962 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.654964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.655002 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.655127 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.655463 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.656117 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.656826 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.657039 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.657124 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.659139 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.659147 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.659150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.659174 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.660630 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.661129 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.661133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.661135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.661219 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.661577 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.661580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.661581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.661762 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.661848 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.662074 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.662176 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":105,"threads_started":1}
I20260808 18:27:54.663007 18035 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.124@KRBTEST.COM for oryx/127.7.164.124@KRBTEST.COM
I20260808 18:27:54.669939 18038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.663487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:38159 (local address 127.0.0.1:50894)
0808 18:27:54.663620 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.663635 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.663713 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.663908 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.663911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.664072 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.664132 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.664137 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.664813 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.664818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.665304 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.665311 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.665742 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.665745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.665816 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.666196 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.666206 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.667355 (+ 1149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.668749 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.668752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.668753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.668938 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.669170 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.669172 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.669174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.669248 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.669550 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.669552 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.669626 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.669705 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.669794 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 18:27:54.669962 18037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.663543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50894 (local address 127.7.164.124:38159)
0808 18:27:54.663629 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.663635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.663677 (+ 42us) server_negotiation.cc:407] Connection header received
0808 18:27:54.663781 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.663785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.663821 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.663897 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.664233 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.664714 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.665411 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.665642 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.665742 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.667471 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.667474 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.667476 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.667496 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.668653 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.669020 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.669022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.669023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.669094 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.669322 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.669324 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.669325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.669474 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.669554 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.669760 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.669845 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 18:27:54.673293 18035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: Bootstrap starting.
I20260808 18:27:54.675719 18035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:54.676491 18035 log.cc:824] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:54.678444 18035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: No bootstrap required, opened a new log
I20260808 18:27:54.681370 18035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } }
I20260808 18:27:54.681609 18035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:54.681682 18035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f431983eab4f48bc40603dca5208f6, State: Initialized, Role: FOLLOWER
I20260808 18:27:54.682101 18035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } }
I20260808 18:27:54.682602 18041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } } }
I20260808 18:27:54.682729 18041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:54.683274 18035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 18:27:54.687963 18060 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:27:54.688047 18060 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:27:54.691561 17799 master_runner.cc:428] Detected that this master 87f431983eab4f48bc40603dca5208f6 is joining an existing cluster
I20260808 18:27:54.691642 17799 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.164.124:38159
I20260808 18:27:54.703673 18037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.694537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50896 (local address 127.7.164.124:38159)
0808 18:27:54.694639 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.694644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.695291 (+ 647us) server_negotiation.cc:407] Connection header received
0808 18:27:54.695432 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.695435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.695479 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.695522 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.696084 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.696762 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.699112 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.699342 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.699452 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.700500 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.700504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.700506 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.700528 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.701932 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.702454 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.702457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.702459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.702536 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.702825 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.702828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.702830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.703012 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.703114 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.703351 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.703491 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":40,"spinlock_wait_cycles":3456}
I20260808 18:27:54.703685 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.694717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:38159 (local address 127.0.0.1:50896)
0808 18:27:54.695209 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.695222 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.695340 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.695623 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.695625 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.695832 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.695975 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.695982 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.696874 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.696877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.698761 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.698771 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.699451 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.699455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.699546 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.699975 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.699995 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.700382 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.702043 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.702049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.702051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.702343 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.702634 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.702637 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.702640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.702731 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.703103 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.703106 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.703189 (+ 83us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.703294 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.703437 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260808 18:27:54.704128 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.693910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53550 (local address 127.7.164.125:43073)
0808 18:27:54.694056 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.694061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.694494 (+ 433us) server_negotiation.cc:407] Connection header received
0808 18:27:54.694687 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.694690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.694745 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.694801 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.695587 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.696478 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.697848 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.698281 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.698379 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.699841 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.699846 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.699849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.699881 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.701941 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.702639 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.702643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.702645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.702740 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.703111 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.703115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.703119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.703368 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.703447 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.703820 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.703959 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:27:54.704694 18069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.694070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:43073 (local address 127.0.0.1:53550)
0808 18:27:54.694398 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.694415 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.694567 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.694968 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.694971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.695271 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.695423 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.695432 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.696642 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.696646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.697684 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.697697 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.698553 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.698557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.698671 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.699094 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.699111 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.699664 (+ 553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.702135 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.702138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.702140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.702508 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.702867 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.702870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.702872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.702991 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.703532 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.703535 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.703628 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.704320 (+ 692us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.704490 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"spinlock_wait_cycles":768,"thread_start_us":153,"threads_started":1}
I20260808 18:27:54.706401 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 17076
I20260808 18:27:54.727061 18068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.693839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.0.0.1:46096)
0808 18:27:54.694199 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.694240 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.694397 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.694823 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.694825 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.695134 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.695302 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.695313 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.696369 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.696372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.697354 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.697365 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.697832 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.697836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.697920 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.698518 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.698532 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.715165 (+ 16633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.724980 (+ 9815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.724984 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.724987 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.725306 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.725696 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.725699 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.725702 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.725829 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.726388 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.726391 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.726518 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.726644 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.726822 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":18,"thread_start_us":152,"threads_started":1}
I20260808 18:27:54.727468 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.693899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46096 (local address 127.7.164.126:35069)
0808 18:27:54.694031 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.694036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.694316 (+ 280us) server_negotiation.cc:407] Connection header received
0808 18:27:54.694542 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.694545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.694605 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.694661 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.695433 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.696253 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.697500 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.697706 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.697820 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.715424 (+ 17604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.715429 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.715433 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.715473 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.724808 (+ 9335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.725458 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.725465 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.725468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.725579 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.725939 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.725945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.725950 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.726276 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.726385 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.726725 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.726888 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:27:54.730536 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:33189
--local_ip_for_outbound_sockets=127.7.164.65
--tserver_master_addrs=127.7.164.126:35069
--webserver_port=33261
--webserver_interface=127.7.164.65
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.65.keytab
--principal=oryx/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.164.126:35069,127.7.164.125:43073
--tserver_master_addrs=127.7.164.126:35069,127.7.164.125:43073,127.7.164.124:38159 with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213673, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.126@KRBTEST.COM for oryx/127.7.164.124@KRBTEST.COM
I20260808 18:27:54.742727 18037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.731516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50904 (local address 127.7.164.124:38159)
0808 18:27:54.731626 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.731631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.731698 (+ 67us) server_negotiation.cc:407] Connection header received
0808 18:27:54.731837 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.731840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.731892 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.731975 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.732470 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.733272 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.734540 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.734917 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.735033 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.737709 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.737714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.737717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.737746 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.740599 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.741158 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.741162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.741164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.741261 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.741601 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.741605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.741607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.741837 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.741950 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.742407 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.742535 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:27:54.742807 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.731454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:38159 (local address 127.0.0.1:50904)
0808 18:27:54.731625 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.731639 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.731726 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.731995 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.731998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.732195 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.732324 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.732334 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.733421 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.733426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.734393 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.734408 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.735119 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.735126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.735260 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.735782 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.735803 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.737532 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.740731 (+ 3199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.740736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.740741 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.741040 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.741370 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.741374 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.741377 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.741493 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.741954 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.741960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.742095 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.742384 (+ 289us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.742571 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 18:27:54.744202 16986 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.164.124:38159: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "87f431983eab4f48bc40603dca5208f6"
member_type: NON_VOTER
last_known_addr {
host: "127.7.164.124"
port: 38159
}
attrs {
promote: true
}
}
dest_uuid: "b0eb5242ae394100a33e0ced51baccb6"
cas_config_opid_index: 5
I20260808 18:27:54.744853 16986 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } }
I20260808 18:27:54.745934 17791 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 87f431983eab4f48bc40603dca5208f6 (127.7.164.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } } }
I20260808 18:27:54.747478 17791 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.164.124:38159
I20260808 18:27:54.747655 17791 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } } }
I20260808 18:27:54.747756 17791 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:54.749039 17995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:54.750551 17995 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 FOLLOWER]: Refusing update from remote peer b0eb5242ae394100a33e0ced51baccb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 18:27:54.751472 17790 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:54.752564 17799 master.cc:537] Master@127.7.164.124:38159 shutting down...
I20260808 18:27:54.756126 17995 raft_consensus.cc:858] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } }
W20260808 18:27:54.756239 17995 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } }
I20260808 18:27:54.758643 18073 mvcc.cc:205] Tried to move back new op lower bound from 7316331206207021056 to 7316331206044160000. Current Snapshot: MvccSnapshot[applied={T|T < 7316331206079516672}]
I20260808 18:27:54.757949 17995 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 49041fa054ed40ca9242a05b3d0eaddb (127.7.164.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } }
I20260808 18:27:54.764000 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.748376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53566 (local address 127.7.164.125:43073)
0808 18:27:54.748536 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.748541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.759240 (+ 10699us) server_negotiation.cc:407] Connection header received
0808 18:27:54.759484 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.759487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.759552 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.759617 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:54.760184 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.760947 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.762476 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.763125 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.763211 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.763370 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.763764 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.763844 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:27:54.764045 18075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } }
I20260808 18:27:54.764144 18075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 18:27:54.764232 18075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } }
I20260808 18:27:54.764295 18075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 18:27:54.764341 18041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0eb5242ae394100a33e0ced51baccb6. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } }
I20260808 18:27:54.764437 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.748146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:43073 (local address 127.0.0.1:53566)
0808 18:27:54.759143 (+ 10997us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.759161 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.759349 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.759922 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.759925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.759946 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:54.760053 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.760059 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.761106 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.761109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.762330 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.762343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.763485 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.763488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.763586 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.763635 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.764211 (+ 576us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.764302 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10831}
I20260808 18:27:54.764453 18041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 18:27:54.764946 17858 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 LEARNER]: Refusing update from remote peer b0eb5242ae394100a33e0ced51baccb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 18:27:54.765547 17788 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 18:27:54.766095 17790 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } } }
I20260808 18:27:54.766251 17790 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:54.767254 17799 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260808 18:27:54.767529 17799 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260808 18:27:54.767606 17799 tablet_replica.cc:330] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: stopping tablet replica
I20260808 18:27:54.768424 17858 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 87f431983eab4f48bc40603dca5208f6 (127.7.164.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } } }
I20260808 18:27:54.770749 17909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0eb5242ae394100a33e0ced51baccb6. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } } }
I20260808 18:27:54.770880 17909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: This master's current role is: LEARNER
I20260808 18:27:54.771003 17909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: NON_VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: true } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } } }
I20260808 18:27:54.771098 17909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: This master's current role is: LEARNER
W20260808 18:27:54.774017 16963 consensus_peers.cc:487] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 -> Peer 87f431983eab4f48bc40603dca5208f6 (127.7.164.124:38159): Couldn't send request to peer 87f431983eab4f48bc40603dca5208f6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 18:27:54.774247 17788 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6: attempting to promote NON_VOTER 49041fa054ed40ca9242a05b3d0eaddb to VOTER
I20260808 18:27:54.774765 17788 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } }
I20260808 18:27:54.776943 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.776647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:38159 (local address 127.0.0.1:50912)
0808 18:27:54.776788 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.776866 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:38159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260808 18:27:54.777096 16963 consensus_peers.cc:487] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 -> Peer 87f431983eab4f48bc40603dca5208f6 (127.7.164.124:38159): Couldn't send request to peer 87f431983eab4f48bc40603dca5208f6. Status: Network error: Client connection negotiation failed: client connection to 127.7.164.124:38159: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 18:27:54.779047 17858 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 LEARNER]: Refusing update from remote peer b0eb5242ae394100a33e0ced51baccb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 18:27:54.779471 17791 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 18:27:54.781507 17791 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 49041fa054ed40ca9242a05b3d0eaddb (127.7.164.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } } }
I20260808 18:27:54.781961 17858 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 49041fa054ed40ca9242a05b3d0eaddb (127.7.164.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } } }
I20260808 18:27:54.783061 17791 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } } }
I20260808 18:27:54.783181 17791 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:54.783332 17791 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } } }
I20260808 18:27:54.783412 17791 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:54.783756 18082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } } }
I20260808 18:27:54.783882 18082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:54.791975 17799 master.cc:552] Master@127.7.164.124:38159 shutdown complete.
I20260808 18:27:54.792040 17799 master_runner.cc:305] Clearing existing system tablet
I20260808 18:27:54.792305 17799 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:54.792379 17799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:54.793812 17799 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 18:27:54.794878 18085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:54.795143 17799 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:54.795285 17799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 18:27:54.795331 17799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/wal
uuid: "87f431983eab4f48bc40603dca5208f6"
format_stamp: "Formatted at 2026-08-08 18:27:54 on dist-test-slave-jztv"
server_key: "def7857055d48d05ae72e1644886ca42"
server_key_iv: "0f15943a77fb0dc9eb8d54da04e3ea9b"
server_key_version: "encryptionkey@0"
I20260808 18:27:54.797350 17799 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:54.800323 17799 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 18:27:54.800444 17799 log.cc:1196] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 18:27:54.800740 17799 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: Deleting consensus metadata
I20260808 18:27:54.800913 17799 master_runner.cc:315] Copying system tablet from 127.7.164.126:35069
I20260808 18:27:54.802088 17799 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: tablet copy: Beginning tablet copy session from remote peer at address 127.7.164.126:35069
I20260808 18:27:54.812580 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.803097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.7.164.126:35069)
0808 18:27:54.803219 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.803223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.803239 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:27:54.803286 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.803289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.803338 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.803441 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.804063 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.804900 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.806272 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.806683 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.806807 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.808962 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.808966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.808969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.808998 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.810714 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.811247 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.811251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.811253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.811346 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.811663 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.811666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.811668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.811894 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.811992 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.812306 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.812426 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:27:54.812633 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.802733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.0.0.1:46112)
0808 18:27:54.802998 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.803015 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.803130 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.803493 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.803497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.803748 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.803890 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.803898 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.805037 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.805041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.806081 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.806093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.806818 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.806822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.806931 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.808247 (+ 1316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.808264 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.808743 (+ 479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.810834 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.810838 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.810841 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.811131 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.811452 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.811455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.811458 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.811562 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.811997 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.812001 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.812114 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.812239 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.812399 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":96,"threads_started":1}
I20260808 18:27:54.813344 17018 tablet_copy_service.cc:140] P b0eb5242ae394100a33e0ced51baccb6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 87f431983eab4f48bc40603dca5208f6 ({username='oryx', principal='oryx/127.7.164.124@KRBTEST.COM'} at 127.0.0.1:46112)
I20260808 18:27:54.813438 17018 tablet_copy_service.cc:161] P b0eb5242ae394100a33e0ced51baccb6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 87f431983eab4f48bc40603dca5208f6 at {username='oryx', principal='oryx/127.7.164.124@KRBTEST.COM'} at 127.0.0.1:46112: session id = 87f431983eab4f48bc40603dca5208f6-00000000000000000000000000000000
I20260808 18:27:54.814603 17018 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 18:27:54.816039 17799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 18:27:54.818522 17799 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: tablet copy: Starting download of 0 data blocks...
I20260808 18:27:54.818711 17799 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: tablet copy: Starting download of 1 WAL segments...
I20260808 18:27:54.820205 17799 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 18:27:54.822118 17018 tablet_copy_service.cc:342] P b0eb5242ae394100a33e0ced51baccb6: Request end of tablet copy session 87f431983eab4f48bc40603dca5208f6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.164.124@KRBTEST.COM'} at 127.0.0.1:46112
I20260808 18:27:54.822203 17018 tablet_copy_service.cc:433] P b0eb5242ae394100a33e0ced51baccb6: ending tablet copy session 87f431983eab4f48bc40603dca5208f6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 87f431983eab4f48bc40603dca5208f6
I20260808 18:27:54.830763 17799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:54.832536 18094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:54.832721 18096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:54.832831 18099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:54.833801 17799 server_base.cc:1034] running on GCE node
I20260808 18:27:54.833972 17799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:54.834287 17799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:54.835443 17799 hybrid_clock.cc:648] HybridClock initialized: now 1786213674835423 us; error 39 us; skew 500 ppm
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:54.837013 17799 init.cc:377] Logged in from keytab as oryx/127.7.164.124@KRBTEST.COM (short username oryx)
I20260808 18:27:54.837464 17799 webserver.cc:460] Webserver started at http://127.7.164.124:39559/ using document root <none> and password file <none>
I20260808 18:27:54.837651 17799 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:54.837693 17799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:54.842185 17799 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 18:27:54.843475 18107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:54.843772 17799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:54.843870 17799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/wal
uuid: "87f431983eab4f48bc40603dca5208f6"
format_stamp: "Formatted at 2026-08-08 18:27:54 on dist-test-slave-jztv"
server_key: "def7857055d48d05ae72e1644886ca42"
server_key_iv: "0f15943a77fb0dc9eb8d54da04e3ea9b"
server_key_version: "encryptionkey@0"
I20260808 18:27:54.843957 17799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:54.861949 17455 heartbeater.cc:503] Master 127.7.164.126:35069 was elected leader, sending a full tablet report...
W20260808 18:27:54.881386 18072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:54.881646 18072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:54.881721 18072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:54.885926 18072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:54.886113 18072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:54.886257 18072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:27:54.886749 17799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:54.889148 17799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:54.893767 18072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.65.keytab
--principal=oryx/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:33189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=33261
--tserver_master_addrs=127.7.164.126:35069,127.7.164.125:43073,127.7.164.124:38159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:54.895176 18072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:54.896395 17799 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.124:38159
I20260808 18:27:54.896452 18072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:27:54.897246 17799 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/master-2/data/info.pb
I20260808 18:27:54.898514 18189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.124:38159 every 8 connection(s)
I20260808 18:27:54.899426 18190 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:27:54.900523 18190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: Bootstrap starting.
W20260808 18:27:54.910341 18195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:54.910521 18072 server_base.cc:1034] running on GCE node
W20260808 18:27:54.910787 18196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:54.911046 18200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:54.911245 18190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:27:54.911504 18190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: Bootstrap complete.
I20260808 18:27:54.911516 18072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:54.911964 18190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } }
I20260808 18:27:54.912065 18190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 87f431983eab4f48bc40603dca5208f6, State: Initialized, Role: LEARNER
I20260808 18:27:54.912143 18190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } }
I20260808 18:27:54.912233 18072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:54.912647 18190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:54.913417 18072 hybrid_clock.cc:648] HybridClock initialized: now 1786213674913350 us; error 80 us; skew 500 ppm
I20260808 18:27:54.913386 18205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } } }
I20260808 18:27:54.913537 18205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: This master's current role is: LEARNER
I20260808 18:27:54.915933 18225 catalog_manager.cc:1259] Loaded cluster ID: 772ce27b4ff4457e9e891cd52b827ba7
I20260808 18:27:54.915987 18225 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: loading cluster ID for follower catalog manager: success
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:54.917024 18072 init.cc:377] Logged in from keytab as oryx/127.7.164.65@KRBTEST.COM (short username oryx)
I20260808 18:27:54.917798 18225 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: acquiring CA information for follower catalog manager: success
I20260808 18:27:54.918026 18072 webserver.cc:460] Webserver started at http://127.7.164.65:33261/ using document root <none> and password file <none>
I20260808 18:27:54.918663 18072 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:54.918769 18072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:54.919193 18225 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:27:54.923254 18072 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:27:54.925542 18234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:54.926872 18072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:27:54.927004 18072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "b328e02ab41241ec9edc85a11d4da0fe"
format_stamp: "Formatted at 2026-08-08 18:27:53 on dist-test-slave-jztv"
server_key: "626cfd7157ca67206342761087f9e907"
server_key_iv: "a74ba4a20fec21cc945e5cef6019658f"
server_key_version: "encryptionkey@0"
I20260808 18:27:54.927670 18072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:54.947815 18072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:54.951022 18072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:54.951315 18072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:54.952111 18072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:54.953130 18072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:54.953197 18072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:54.953265 18072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:54.953310 18072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:54.967803 18072 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:33189
I20260808 18:27:54.967809 18397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:33189 every 8 connection(s)
I20260808 18:27:54.968878 18072 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:27:54.969681 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 18072
I20260808 18:27:54.969918 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 17267
I20260808 18:27:54.988435 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:37629
--local_ip_for_outbound_sockets=127.7.164.66
--tserver_master_addrs=127.7.164.126:35069
--webserver_port=39091
--webserver_interface=127.7.164.66
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.66.keytab
--principal=oryx/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.164.126:35069,127.7.164.125:43073
--tserver_master_addrs=127.7.164.126:35069,127.7.164.125:43073,127.7.164.124:38159 with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.125@KRBTEST.COM
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
Aug 08 18:27:54 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213674, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.124@KRBTEST.COM
I20260808 18:27:54.995527 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.980854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:42349 (local address 127.7.164.126:35069)
0808 18:27:54.981021 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.981025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.981038 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:27:54.981080 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.981082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.981126 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.981214 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.982891 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.983717 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.985975 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.986356 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.986446 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.990599 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.990605 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.990609 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.990647 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.993436 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.994062 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.994067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.994070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.994168 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.994592 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.994596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.994598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.994809 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.994922 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.995247 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.995364 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 18:27:54.995527 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.978702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:47205 (local address 127.7.164.125:43073)
0808 18:27:54.980414 (+ 1712us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.980419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.980435 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:27:54.980484 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.980487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.980539 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.980634 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.982902 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.983716 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.985975 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.986349 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.986437 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.990933 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.990937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.990939 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.990967 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.992697 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.993479 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.993482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.993484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.993734 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.994042 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.994045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.994047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.994287 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.994381 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.995196 (+ 815us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.995350 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1619}
I20260808 18:27:54.996672 18406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.978602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:43073 (local address 127.7.164.65:47205)
0808 18:27:54.979639 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.979662 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.979819 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.981093 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.981102 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.981989 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.982700 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.982724 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.983956 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.983959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.985831 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.985838 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.986617 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.986620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.986974 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.988066 (+ 1092us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.988085 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.990761 (+ 2676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.992863 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.992870 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.992886 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.993354 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.993845 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.993847 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.993849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.993936 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.994451 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.994453 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.994875 (+ 422us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.996105 (+ 1230us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.996241 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"mutex_wait_us":489,"spinlock_wait_cycles":6656,"thread_start_us":714,"threads_started":1}
I20260808 18:27:54.996676 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.975945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.65:42349)
0808 18:27:54.976988 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.977043 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.978147 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.981217 (+ 3070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.981223 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.981760 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.982702 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.982724 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.983962 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.983965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.985825 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.985838 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.986617 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.986620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.986973 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.987799 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.987825 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.990386 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.993607 (+ 3221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.993614 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.993619 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.993911 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.994345 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.994350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.994353 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.994456 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.994970 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.994973 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.995057 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.995678 (+ 621us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.995877 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"mutex_wait_us":485,"spinlock_wait_cycles":17792,"thread_start_us":637,"threads_started":1}
I20260808 18:27:54.997241 18411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.980472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:56051 (local address 127.7.164.124:38159)
0808 18:27:54.980741 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:54.980757 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:54.980776 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:27:54.980825 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:54.980830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:54.980888 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:54.980980 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:54.983379 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.984532 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.986034 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.986481 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.986607 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.990924 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:54.990928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:54.990930 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:54.990960 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:54.992874 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.993478 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.993481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.993483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.993588 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:54.994010 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:54.994016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:54.994021 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:54.994289 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:54.994381 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:54.996816 (+ 2435us) server_negotiation.cc:299] Negotiation successful
0808 18:27:54.996976 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":83,"threads_started":1}
I20260808 18:27:54.996680 18410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:54.975945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:38159 (local address 127.7.164.65:56051)
0808 18:27:54.979503 (+ 3558us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:54.979520 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:54.979626 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:54.981132 (+ 1506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:54.981138 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:54.982942 (+ 1804us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:54.983209 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.983223 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.984683 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.984689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.985867 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:54.985882 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:54.986753 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:54.986760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:54.987020 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:54.987944 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:54.987961 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:54.990760 (+ 2799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:54.992993 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.992996 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.992998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.993354 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.993676 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:54.993681 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:54.993686 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:54.993891 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:54.994444 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:54.994451 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:54.994885 (+ 434us) client_negotiation.cc:769] Sending connection context
0808 18:27:54.995123 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 18:27:54.995543 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3441,"mutex_wait_us":1584,"spinlock_wait_cycles":15104,"thread_start_us":258,"threads_started":1}
I20260808 18:27:54.998265 18399 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35069
I20260808 18:27:54.998531 18399 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:54.998528 18400 heartbeater.cc:347] Connected to a master server at 127.7.164.125:43073
I20260808 18:27:54.998672 18400 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:54.999166 18398 heartbeater.cc:347] Connected to a master server at 127.7.164.124:38159
I20260808 18:27:54.999253 18398 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:54.999240 18399 heartbeater.cc:511] Master 127.7.164.126:35069 requested a full tablet report, sending...
I20260808 18:27:54.999472 18398 heartbeater.cc:511] Master 127.7.164.124:38159 requested a full tablet report, sending...
I20260808 18:27:54.999235 18400 heartbeater.cc:511] Master 127.7.164.125:43073 requested a full tablet report, sending...
I20260808 18:27:55.000718 16986 ts_manager.cc:194] Re-registered known tserver with Master: b328e02ab41241ec9edc85a11d4da0fe (127.7.164.65:33189)
I20260808 18:27:55.000924 17847 ts_manager.cc:194] Registered new tserver with Master: b328e02ab41241ec9edc85a11d4da0fe (127.7.164.65:33189)
I20260808 18:27:55.000952 18135 ts_manager.cc:194] Registered new tserver with Master: b328e02ab41241ec9edc85a11d4da0fe (127.7.164.65:33189)
I20260808 18:27:55.001492 16986 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:42349
I20260808 18:27:55.094342 17646 heartbeater.cc:503] Master 127.7.164.126:35069 was elected leader, sending a full tablet report...
W20260808 18:27:55.108656 18412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:55.108937 18412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:55.109022 18412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:55.114131 18412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:55.114216 18412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:55.114332 18412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:27:55.119845 18412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.66.keytab
--principal=oryx/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:37629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=39091
--tserver_master_addrs=127.7.164.126:35069,127.7.164.125:43073,127.7.164.124:38159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:55.121007 18412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:55.122054 18412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:55.131831 18421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:55.131932 18418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:55.131836 18419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:55.132328 18412 server_base.cc:1034] running on GCE node
I20260808 18:27:55.132743 18412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:55.133349 18412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:55.134526 18412 hybrid_clock.cc:648] HybridClock initialized: now 1786213675134512 us; error 34 us; skew 500 ppm
Aug 08 18:27:55 dist-test-slave-jztv krb5kdc[16918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213675, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:55.137208 18412 init.cc:377] Logged in from keytab as oryx/127.7.164.66@KRBTEST.COM (short username oryx)
I20260808 18:27:55.138096 18412 webserver.cc:460] Webserver started at http://127.7.164.66:39091/ using document root <none> and password file <none>
I20260808 18:27:55.138700 18412 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:55.138808 18412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:55.142670 18412 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:27:55.144784 18431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:55.145994 18412 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:55.146127 18412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "49a0cb2e3dad4a548936000ec3d80547"
format_stamp: "Formatted at 2026-08-08 18:27:53 on dist-test-slave-jztv"
server_key: "7ed992c131e832abe1343fe6828babe3"
server_key_iv: "f65fc7a123b920ae0369ad69e551631d"
server_key_version: "encryptionkey@0"
I20260808 18:27:55.146729 18412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:55.162791 18412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:55.165890 18412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:55.166086 18412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:55.166723 18412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:55.167580 18412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:55.167647 18412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:55.167717 18412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:55.167757 18412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:55.177954 18412 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:37629
I20260808 18:27:55.178015 18601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:37629 every 8 connection(s)
I20260808 18:27:55.179319 18412 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:27:55.182871 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.178316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:38159 (local address 127.0.0.1:50922)
0808 18:27:55.178437 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:55.178455 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:55.178559 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:55.178873 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:55.178875 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:55.178890 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:55.178978 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.178983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.180068 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.180074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.181246 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.181263 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.182452 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.182456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.182540 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.182565 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:27:55.182640 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 18:27:55.182730 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 18:27:55.183315 18411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.178393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50922 (local address 127.7.164.124:38159)
0808 18:27:55.178530 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.178535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.178555 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:27:55.178671 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.178674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.178741 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.178866 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:55.179091 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.179904 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.181403 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.182058 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.182168 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.182383 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.183119 (+ 736us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.183202 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:27:55.184161 18149 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 18:27:55.184430 18205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0eb5242ae394100a33e0ced51baccb6. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } } }
I20260808 18:27:55.184573 18205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: This master's current role is: LEARNER
I20260808 18:27:55.186887 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 18412
I20260808 18:27:55.187074 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 17462
Aug 08 18:27:55 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213675, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.124@KRBTEST.COM
Aug 08 18:27:55 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213675, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.125@KRBTEST.COM
Aug 08 18:27:55 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213675, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
I20260808 18:27:55.209018 18411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.181887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:49471 (local address 127.7.164.124:38159)
0808 18:27:55.183506 (+ 1619us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.183510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.183524 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:27:55.183557 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.183560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.183606 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.183693 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:55.185303 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.186066 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.199609 (+ 13543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.200085 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.200184 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.203984 (+ 3800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:55.203991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:55.203996 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:55.204029 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:55.206065 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.206816 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.206820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.206823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.206925 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.207393 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.207396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.207398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.207673 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:55.207767 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.208629 (+ 862us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.208784 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1544,"thread_start_us":83,"threads_started":1}
I20260808 18:27:55.209028 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.183076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:42637 (local address 127.7.164.125:43073)
0808 18:27:55.183183 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.183187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.183862 (+ 675us) server_negotiation.cc:407] Connection header received
0808 18:27:55.184770 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.184773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.184824 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.184901 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:55.185362 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.185850 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.201726 (+ 15876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.202167 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.202319 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.205461 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:55.205465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:55.205467 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:55.205497 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:55.207216 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.207746 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.207749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.207751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.207842 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.208139 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.208142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.208144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.208357 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:55.208425 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.208756 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.208880 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:27:55.210795 18609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.182753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:43073 (local address 127.7.164.66:42637)
0808 18:27:55.183755 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:55.183772 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:55.183898 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:55.184928 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:55.184933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:55.185106 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:55.185216 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.185226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.185999 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.186001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.201509 (+ 15508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.201524 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.202603 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.202608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.202724 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.203409 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:55.203427 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:55.205277 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:55.207362 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:55.207366 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:55.207368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:55.207623 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:55.207960 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:55.207962 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:55.207964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:55.208046 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:55.208499 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:55.208502 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:55.208587 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 18:27:55.209210 (+ 623us) client_negotiation.cc:240] Negotiation successful
0808 18:27:55.209640 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":8,"thread_start_us":81,"threads_started":1}
I20260808 18:27:55.211690 18606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.181690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:38159 (local address 127.7.164.66:49471)
0808 18:27:55.182245 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:55.182295 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:55.183293 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:55.183859 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:55.183866 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:55.184383 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:55.185165 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.185184 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.186201 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.186204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.199287 (+ 13083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.199319 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.200435 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.200452 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.200725 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.201559 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:55.201586 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:55.203771 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:55.206248 (+ 2477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:55.206255 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:55.206274 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:55.206674 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:55.207055 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:55.207060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:55.207064 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:55.207270 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:55.207840 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:55.207847 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:55.208209 (+ 362us) client_negotiation.cc:769] Sending connection context
0808 18:27:55.209762 (+ 1553us) client_negotiation.cc:240] Negotiation successful
0808 18:27:55.209933 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":100,"thread_start_us":145,"threads_started":1}
I20260808 18:27:55.212445 18608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.181690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.66:37107)
0808 18:27:55.184095 (+ 2405us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:55.184106 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:55.184214 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:55.184551 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:55.184554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:55.184776 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:55.185162 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.185185 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.186179 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.186182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.205617 (+ 19435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.205632 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.206521 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.206527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.206629 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.207112 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:55.207127 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:55.208556 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:55.210669 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:55.210672 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:55.210674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:55.210937 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:55.211285 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:55.211287 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:55.211289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:55.211388 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:55.211917 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:55.211920 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:55.212025 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 18:27:55.212139 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 18:27:55.212276 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2329,"mutex_wait_us":1755,"thread_start_us":117,"threads_started":1}
I20260808 18:27:55.212484 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.181134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:37107 (local address 127.7.164.126:35069)
0808 18:27:55.181239 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.181243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.184190 (+ 2947us) server_negotiation.cc:407] Connection header received
0808 18:27:55.184314 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.184318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.184367 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.184437 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:55.185339 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.186072 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.205801 (+ 19729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.206210 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.206321 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.208712 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:55.208716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:55.208719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:55.208751 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:55.210531 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.211071 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.211074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.211077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.211169 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.211497 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.211501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.211504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.211757 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:55.211846 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.212216 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.212331 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:27:55.213071 18603 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35069
I20260808 18:27:55.213289 18604 heartbeater.cc:347] Connected to a master server at 127.7.164.125:43073
I20260808 18:27:55.213356 18604 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:55.213341 18603 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:55.213734 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:42685
--local_ip_for_outbound_sockets=127.7.164.67
--tserver_master_addrs=127.7.164.126:35069
--webserver_port=43183
--webserver_interface=127.7.164.67
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.67.keytab
--principal=oryx/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.164.126:35069,127.7.164.125:43073
--tserver_master_addrs=127.7.164.126:35069,127.7.164.125:43073,127.7.164.124:38159 with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:27:55.213991 18604 heartbeater.cc:511] Master 127.7.164.125:43073 requested a full tablet report, sending...
I20260808 18:27:55.213991 18603 heartbeater.cc:511] Master 127.7.164.126:35069 requested a full tablet report, sending...
I20260808 18:27:55.215274 18602 heartbeater.cc:347] Connected to a master server at 127.7.164.124:38159
I20260808 18:27:55.215366 18602 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:55.215543 18602 heartbeater.cc:511] Master 127.7.164.124:38159 requested a full tablet report, sending...
I20260808 18:27:55.215549 16986 ts_manager.cc:194] Re-registered known tserver with Master: 49a0cb2e3dad4a548936000ec3d80547 (127.7.164.66:37629)
I20260808 18:27:55.216068 17847 ts_manager.cc:194] Registered new tserver with Master: 49a0cb2e3dad4a548936000ec3d80547 (127.7.164.66:37629)
I20260808 18:27:55.216255 16986 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:37107
I20260808 18:27:55.216357 18135 ts_manager.cc:194] Registered new tserver with Master: 49a0cb2e3dad4a548936000ec3d80547 (127.7.164.66:37629)
W20260808 18:27:55.333092 18610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:55.333371 18610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:55.333442 18610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:55.337560 18610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:55.337630 18610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:55.337723 18610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:27:55.343204 18610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.67.keytab
--principal=oryx/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:42685
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=43183
--tserver_master_addrs=127.7.164.126:35069,127.7.164.125:43073,127.7.164.124:38159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:55.344385 18610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:55.345515 18610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:55.355872 18617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:55.355798 18616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:55.355815 18622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:55.356127 18610 server_base.cc:1034] running on GCE node
I20260808 18:27:55.356568 18610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:55.357200 18610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:55.358371 18610 hybrid_clock.cc:648] HybridClock initialized: now 1786213675358343 us; error 51 us; skew 500 ppm
Aug 08 18:27:55 dist-test-slave-jztv krb5kdc[16918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213675, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:55.361255 18610 init.cc:377] Logged in from keytab as oryx/127.7.164.67@KRBTEST.COM (short username oryx)
I20260808 18:27:55.362106 18610 webserver.cc:460] Webserver started at http://127.7.164.67:43183/ using document root <none> and password file <none>
I20260808 18:27:55.362862 18610 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:55.362943 18610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:55.366700 18610 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 18:27:55.368657 18630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:55.369870 18610 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:55.369987 18610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "d3db78488e4f4f7489023ebb7002a030"
format_stamp: "Formatted at 2026-08-08 18:27:54 on dist-test-slave-jztv"
server_key: "321e8575687fff667dbdd1665ce3cc43"
server_key_iv: "b625eccd618132bef0cf58688ba629d4"
server_key_version: "encryptionkey@0"
I20260808 18:27:55.370589 18610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:55.387028 18610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:55.390317 18610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:55.390513 18610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:55.391135 18610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:27:55.392006 18610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:55.392055 18610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:55.392118 18610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:55.392148 18610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:55.402200 18610 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:42685
I20260808 18:27:55.402247 18803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:42685 every 8 connection(s)
I20260808 18:27:55.403358 18610 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:27:55.411952 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 18610
Aug 08 18:27:55 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213675, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
Aug 08 18:27:55 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213675, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.125@KRBTEST.COM
Aug 08 18:27:55 dist-test-slave-jztv krb5kdc[16918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213675, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.124@KRBTEST.COM
I20260808 18:27:55.421564 18814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.406820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:43073 (local address 127.7.164.67:38887)
0808 18:27:55.407112 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:55.407126 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:55.407416 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:55.407953 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:55.407960 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:55.408456 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:55.409019 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.409029 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.410138 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.410142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.411321 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.411332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.412594 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.412597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.412694 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.413230 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:55.413253 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:55.415720 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:55.417927 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:55.417935 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:55.417953 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:55.418357 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:55.418684 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:55.418688 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:55.418689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:55.418878 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:55.419384 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:55.419391 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:55.419742 (+ 351us) client_negotiation.cc:769] Sending connection context
0808 18:27:55.419949 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 18:27:55.420334 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":231,"spinlock_wait_cycles":51328,"thread_start_us":91,"threads_started":1}
I20260808 18:27:55.422374 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.407169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:38887 (local address 127.7.164.125:43073)
0808 18:27:55.407296 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.407299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.407313 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:27:55.407658 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.407662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.407720 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.407775 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:55.409177 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.409987 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.412066 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.412458 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.412571 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.415970 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:55.415974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:55.415977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:55.416007 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:55.417785 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.418476 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.418480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.418481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.418577 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.419009 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.419013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.419014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.419226 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:55.419311 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.422022 (+ 2711us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.422157 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:27:55.422657 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.405272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:38743 (local address 127.7.164.126:35069)
0808 18:27:55.405463 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.405469 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.406437 (+ 968us) server_negotiation.cc:407] Connection header received
0808 18:27:55.407626 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.407629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.407692 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.407773 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:55.409151 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.409687 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.410919 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.411141 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.411246 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.418428 (+ 7182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:55.418432 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:55.418435 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:55.418465 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:55.420191 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.420969 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.420972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.420974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.421062 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.421397 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.421400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.421402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.421848 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:55.421942 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.422357 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.422501 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 18:27:55.423349 18808 heartbeater.cc:347] Connected to a master server at 127.7.164.125:43073
I20260808 18:27:55.423656 18808 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:55.424302 18808 heartbeater.cc:511] Master 127.7.164.125:43073 requested a full tablet report, sending...
I20260808 18:27:55.424835 18411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.405316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:46369 (local address 127.7.164.124:38159)
0808 18:27:55.405447 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.405452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.406440 (+ 988us) server_negotiation.cc:407] Connection header received
0808 18:27:55.407658 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.407662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.407720 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.407775 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:55.409175 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.409987 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.411457 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.411797 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.411940 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.416168 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:55.416173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:55.416176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:55.416208 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:55.418019 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.423275 (+ 5256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.423279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.423281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.423396 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.423742 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.423745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.423747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.424010 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:55.424108 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.424519 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.424654 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:27:55.425412 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.405756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.67:38743)
0808 18:27:55.406274 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:55.406332 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:55.407416 (+ 1084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:55.407951 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:55.407958 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:55.408422 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:55.409006 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.409021 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.409830 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.409833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.410795 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.410805 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.411301 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.411305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.411535 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.412386 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:55.412408 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:55.417196 (+ 4788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:55.420313 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:55.420316 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:55.420318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:55.420855 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:55.421169 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:55.421173 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:55.421177 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:55.421271 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:55.422005 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:55.422007 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:55.422109 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 18:27:55.422855 (+ 746us) client_negotiation.cc:240] Negotiation successful
0808 18:27:55.425192 (+ 2337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":87,"spinlock_wait_cycles":32640,"thread_start_us":171,"threads_started":1}
I20260808 18:27:55.425714 17847 ts_manager.cc:194] Registered new tserver with Master: d3db78488e4f4f7489023ebb7002a030 (127.7.164.67:42685)
I20260808 18:27:55.426059 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.405756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:38159 (local address 127.7.164.67:46369)
0808 18:27:55.406274 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:55.406332 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:55.407416 (+ 1084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:55.407953 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:55.407960 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:55.408435 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:55.409020 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.409029 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.410138 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.410142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.411321 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.411332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.411940 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.411943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.412039 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.413308 (+ 1269us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:55.413323 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:55.415994 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:55.418190 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:55.418193 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:55.418195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:55.423152 (+ 4957us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:55.423526 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:55.423531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:55.423535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:55.423625 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:55.424177 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:55.424182 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:55.424281 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 18:27:55.425048 (+ 767us) client_negotiation.cc:240] Negotiation successful
0808 18:27:55.425855 (+ 807us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":886,"spinlock_wait_cycles":32128,"thread_start_us":121,"threads_started":1}
I20260808 18:27:55.426100 18807 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35069
I20260808 18:27:55.426169 18807 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:55.426369 18807 heartbeater.cc:511] Master 127.7.164.126:35069 requested a full tablet report, sending...
I20260808 18:27:55.426960 18805 heartbeater.cc:347] Connected to a master server at 127.7.164.124:38159
I20260808 18:27:55.427088 18805 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:55.427281 18805 heartbeater.cc:511] Master 127.7.164.124:38159 requested a full tablet report, sending...
I20260808 18:27:55.427412 16986 ts_manager.cc:194] Re-registered known tserver with Master: d3db78488e4f4f7489023ebb7002a030 (127.7.164.67:42685)
I20260808 18:27:55.427997 18135 ts_manager.cc:194] Registered new tserver with Master: d3db78488e4f4f7489023ebb7002a030 (127.7.164.67:42685)
I20260808 18:27:55.428126 16986 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:38743
I20260808 18:27:55.430064 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.414368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44150 (local address 127.7.164.126:35069)
0808 18:27:55.416232 (+ 1864us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.416255 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.416272 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:27:55.416317 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.416321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.416371 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.417510 (+ 1139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:55.417639 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.418486 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.419660 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.419996 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.420461 (+ 465us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.421758 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:55.421762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:55.421764 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:55.421792 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:55.427934 (+ 6142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.428785 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.428791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.428795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.428887 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.429176 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.429180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.429182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.429360 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:55.429509 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.429742 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.429864 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1768,"thread_start_us":102,"threads_started":1}
I20260808 18:27:55.432355 18607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.415454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35410 (local address 127.7.164.124:38159)
0808 18:27:55.415585 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.415590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.417094 (+ 1504us) server_negotiation.cc:407] Connection header received
0808 18:27:55.423256 (+ 6162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.423262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.423323 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.423392 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:55.424242 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.424969 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.426022 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.426388 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.426466 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.428540 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:55.428546 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:55.428550 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:55.428579 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:55.430388 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.430943 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.430946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.430948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.431044 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.431357 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.431363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.431368 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.431614 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:55.431753 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.432076 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.432206 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 18:27:55.433933 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.414651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52744 (local address 127.7.164.125:43073)
0808 18:27:55.416029 (+ 1378us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.416033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.416048 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:27:55.416211 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.416215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.416776 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.417302 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:55.418373 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.419550 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.420768 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.421153 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.421246 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.424624 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:55.424629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:55.424632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:55.424655 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:55.431622 (+ 6967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.432211 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.432215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.432217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.432318 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.432804 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.432808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.432809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.433045 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:55.433291 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.433582 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.433721 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1295,"thread_start_us":104,"threads_started":1}
I20260808 18:27:55.433145 16986 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:27:55.436379 16986 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:27:55.439379 17788 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: NON_VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: true } } }
I20260808 18:27:55.439570 17788 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:55.441380 17788 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6: attempting to promote NON_VOTER 87f431983eab4f48bc40603dca5208f6 to VOTER
I20260808 18:27:55.441758 17788 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } }
I20260808 18:27:55.443125 17858 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 FOLLOWER]: Refusing update from remote peer b0eb5242ae394100a33e0ced51baccb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 18:27:55.443125 18149 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 LEARNER]: Refusing update from remote peer b0eb5242ae394100a33e0ced51baccb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 18:27:55.443504 17788 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 18:27:55.443645 18832 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 18:27:55.445756 18833 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 87f431983eab4f48bc40603dca5208f6 (127.7.164.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } } }
I20260808 18:27:55.446008 17858 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 87f431983eab4f48bc40603dca5208f6 (127.7.164.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } } }
I20260808 18:27:55.445971 18149 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 87f431983eab4f48bc40603dca5208f6 (127.7.164.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } } }
I20260808 18:27:55.447432 18833 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } } }
I20260808 18:27:55.447539 18833 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:55.447682 18833 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } } }
I20260808 18:27:55.447777 18833 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:55.448048 18833 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } } }
I20260808 18:27:55.448165 18833 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:55.448514 18205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } } }
I20260808 18:27:55.448644 18205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:55.448865 18844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b0eb5242ae394100a33e0ced51baccb6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } } }
I20260808 18:27:55.448993 18844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:55.459606 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.454382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:33189 (local address 127.0.0.1:54042)
0808 18:27:55.454597 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:55.454614 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:55.454698 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:55.455398 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:55.455401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:55.455420 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:55.455548 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.455556 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.456824 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.456827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.458071 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.458081 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.459205 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.459209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.459291 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.459346 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 18:27:55.459397 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 18:27:55.459463 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":23}
I20260808 18:27:55.460494 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.454696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.7.164.65:33189)
0808 18:27:55.454954 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.454974 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.454998 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:27:55.455081 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.455085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.455257 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.455433 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:55.455666 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.456684 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.458199 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.459063 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.460006 (+ 943us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.460180 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.460241 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.460356 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":84,"threads_started":1}
I20260808 18:27:55.460803 18849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.454987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:37629 (local address 127.0.0.1:36700)
0808 18:27:55.455249 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:55.455263 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:55.455354 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:55.456280 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:55.456285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:55.456308 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:55.456406 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.456413 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.458082 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.458084 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.459268 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.459277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.460402 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.460406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.460498 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.460535 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 18:27:55.460606 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 18:27:55.460673 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":138,"threads_started":1}
I20260808 18:27:55.461318 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.455453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36700 (local address 127.7.164.66:37629)
0808 18:27:55.455780 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.455793 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.455818 (+ 25us) server_negotiation.cc:407] Connection header received
0808 18:27:55.455897 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.455903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.456119 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.456588 (+ 469us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:55.456787 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.457956 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.459412 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.460267 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.460367 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.460495 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.461079 (+ 584us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.461180 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":137,"threads_started":1}
I20260808 18:27:55.461910 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.455906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:42685 (local address 127.0.0.1:46440)
0808 18:27:55.456166 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:55.456181 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:55.456266 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:55.457132 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:55.457135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:55.457155 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:55.458257 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.458266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.459360 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.459364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.460433 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.460442 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.461559 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.461562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.461649 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.461673 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:27:55.461722 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 18:27:55.461787 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":2,"thread_start_us":102,"threads_started":1}
I20260808 18:27:55.463068 18853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.456366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46440 (local address 127.7.164.67:42685)
0808 18:27:55.456682 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.456694 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.456716 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:27:55.456795 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.456803 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.456980 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.457131 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:55.458356 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.459219 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.460593 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.461398 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.462175 (+ 777us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.462511 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.462564 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.462653 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":108,"threads_started":1}
I20260808 18:27:55.464128 18499 tablet_service.cc:1467] Processing CreateTablet for tablet 9272df1c4de344999296c99c443db182 (DEFAULT_TABLE table=test-table [id=5960d6399cf24519b290b441615f04d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:55.464128 18303 tablet_service.cc:1467] Processing CreateTablet for tablet 9272df1c4de344999296c99c443db182 (DEFAULT_TABLE table=test-table [id=5960d6399cf24519b290b441615f04d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:55.465312 18499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9272df1c4de344999296c99c443db182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:55.465312 18303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9272df1c4de344999296c99c443db182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:55.465433 18702 tablet_service.cc:1467] Processing CreateTablet for tablet 9272df1c4de344999296c99c443db182 (DEFAULT_TABLE table=test-table [id=5960d6399cf24519b290b441615f04d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:55.466470 18702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9272df1c4de344999296c99c443db182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:55.473222 18854 tablet_bootstrap.cc:492] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030: Bootstrap starting.
I20260808 18:27:55.474608 18855 tablet_bootstrap.cc:492] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe: Bootstrap starting.
I20260808 18:27:55.474587 18856 tablet_bootstrap.cc:492] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547: Bootstrap starting.
I20260808 18:27:55.476063 18854 tablet_bootstrap.cc:654] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:55.476781 18855 tablet_bootstrap.cc:654] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:55.476781 18856 tablet_bootstrap.cc:654] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:55.476923 18854 log.cc:824] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:55.477630 18856 log.cc:824] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:55.477630 18855 log.cc:824] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe: Log is configured to *not* fsync() on all Append() calls
I20260808 18:27:55.479202 18854 tablet_bootstrap.cc:492] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030: No bootstrap required, opened a new log
I20260808 18:27:55.479435 18854 ts_tablet_manager.cc:1397] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:27:55.479471 18856 tablet_bootstrap.cc:492] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547: No bootstrap required, opened a new log
I20260808 18:27:55.479676 18856 ts_tablet_manager.cc:1397] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:27:55.479773 18855 tablet_bootstrap.cc:492] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe: No bootstrap required, opened a new log
I20260808 18:27:55.479959 18855 ts_tablet_manager.cc:1397] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:27:55.483340 18856 raft_consensus.cc:348] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } } peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } }
I20260808 18:27:55.483646 18856 raft_consensus.cc:374] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:55.483702 18856 raft_consensus.cc:729] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a0cb2e3dad4a548936000ec3d80547, State: Initialized, Role: FOLLOWER
I20260808 18:27:55.484189 18856 consensus_queue.cc:260] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } } peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } }
I20260808 18:27:55.485241 18856 ts_tablet_manager.cc:1428] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:27:55.485498 18603 heartbeater.cc:503] Master 127.7.164.126:35069 was elected leader, sending a full tablet report...
I20260808 18:27:55.485369 18854 raft_consensus.cc:348] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } } peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } }
I20260808 18:27:55.485661 18854 raft_consensus.cc:374] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:55.485715 18854 raft_consensus.cc:729] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3db78488e4f4f7489023ebb7002a030, State: Initialized, Role: FOLLOWER
I20260808 18:27:55.486021 18855 raft_consensus.cc:348] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } } peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } }
I20260808 18:27:55.486179 18854 consensus_queue.cc:260] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } } peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } }
I20260808 18:27:55.486286 18855 raft_consensus.cc:374] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:55.486335 18855 raft_consensus.cc:729] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b328e02ab41241ec9edc85a11d4da0fe, State: Initialized, Role: FOLLOWER
I20260808 18:27:55.486737 18855 consensus_queue.cc:260] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } } peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } }
I20260808 18:27:55.487248 18854 ts_tablet_manager.cc:1428] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260808 18:27:55.487624 18855 ts_tablet_manager.cc:1428] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260808 18:27:55.488874 18399 heartbeater.cc:503] Master 127.7.164.126:35069 was elected leader, sending a full tablet report...
I20260808 18:27:55.488924 18807 heartbeater.cc:503] Master 127.7.164.126:35069 was elected leader, sending a full tablet report...
I20260808 18:27:55.646811 18860 raft_consensus.cc:482] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:27:55.647063 18860 raft_consensus.cc:504] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } } peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } }
I20260808 18:27:55.648258 18860 leader_election.cc:288] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b328e02ab41241ec9edc85a11d4da0fe (127.7.164.65:33189), d3db78488e4f4f7489023ebb7002a030 (127.7.164.67:42685)
I20260808 18:27:55.653491 18853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.648654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:46003 (local address 127.7.164.67:42685)
0808 18:27:55.648783 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.648788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.648834 (+ 46us) server_negotiation.cc:407] Connection header received
0808 18:27:55.649010 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.649013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.649075 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.649162 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:55.649636 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.650472 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.652208 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.652813 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.652940 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.653016 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.653282 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.653359 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:27:55.653491 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.648654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:51589 (local address 127.7.164.65:33189)
0808 18:27:55.648783 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.648788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.648834 (+ 46us) server_negotiation.cc:407] Connection header received
0808 18:27:55.649010 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.649013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.649065 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.649162 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:55.649636 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.650502 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.652208 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.652817 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.652940 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.653016 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.653282 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.653357 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:27:55.653625 18606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.648550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:42685 (local address 127.7.164.66:46003)
0808 18:27:55.648765 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:55.648782 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:55.648894 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:55.649211 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:55.649215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:55.649247 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:55.649505 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.649512 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.650593 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.650597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.652057 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.652068 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.652941 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.652945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.653122 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.653167 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 18:27:55.653249 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 18:27:55.653368 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":194,"spinlock_wait_cycles":21760}
I20260808 18:27:55.653538 18608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.648549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:33189 (local address 127.7.164.66:51589)
0808 18:27:55.648765 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:55.648782 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:55.648894 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:55.649211 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:55.649215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:55.649246 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:55.649504 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.649512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.650598 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.650601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.652057 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:55.652068 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.652941 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.652945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:55.653125 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.653176 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 18:27:55.653249 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 18:27:55.653369 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":308,"spinlock_wait_cycles":17536}
I20260808 18:27:55.654402 18727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9272df1c4de344999296c99c443db182" candidate_uuid: "49a0cb2e3dad4a548936000ec3d80547" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3db78488e4f4f7489023ebb7002a030" is_pre_election: true
I20260808 18:27:55.654740 18727 raft_consensus.cc:2454] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49a0cb2e3dad4a548936000ec3d80547 in term 0.
W20260808 18:27:55.655277 18811 tablet.cc:2369] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:27:55.655258 18332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9272df1c4de344999296c99c443db182" candidate_uuid: "49a0cb2e3dad4a548936000ec3d80547" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b328e02ab41241ec9edc85a11d4da0fe" is_pre_election: true
I20260808 18:27:55.655300 18433 leader_election.cc:302] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49a0cb2e3dad4a548936000ec3d80547, d3db78488e4f4f7489023ebb7002a030; no voters:
I20260808 18:27:55.655524 18332 raft_consensus.cc:2454] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49a0cb2e3dad4a548936000ec3d80547 in term 0.
I20260808 18:27:55.655619 18860 raft_consensus.cc:2781] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:27:55.655709 18860 raft_consensus.cc:482] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:27:55.655751 18860 raft_consensus.cc:3037] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:55.656921 18860 raft_consensus.cc:504] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } } peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } }
I20260808 18:27:55.657235 18860 leader_election.cc:288] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [CANDIDATE]: Term 1 election: Requested vote from peers b328e02ab41241ec9edc85a11d4da0fe (127.7.164.65:33189), d3db78488e4f4f7489023ebb7002a030 (127.7.164.67:42685)
I20260808 18:27:55.657639 18332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9272df1c4de344999296c99c443db182" candidate_uuid: "49a0cb2e3dad4a548936000ec3d80547" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b328e02ab41241ec9edc85a11d4da0fe"
I20260808 18:27:55.657639 18727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9272df1c4de344999296c99c443db182" candidate_uuid: "49a0cb2e3dad4a548936000ec3d80547" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3db78488e4f4f7489023ebb7002a030"
I20260808 18:27:55.657770 18727 raft_consensus.cc:3037] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:55.657770 18332 raft_consensus.cc:3037] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:55.659088 18727 raft_consensus.cc:2454] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49a0cb2e3dad4a548936000ec3d80547 in term 1.
I20260808 18:27:55.659088 18332 raft_consensus.cc:2454] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49a0cb2e3dad4a548936000ec3d80547 in term 1.
I20260808 18:27:55.659442 18433 leader_election.cc:302] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 49a0cb2e3dad4a548936000ec3d80547, b328e02ab41241ec9edc85a11d4da0fe, d3db78488e4f4f7489023ebb7002a030; no voters:
I20260808 18:27:55.659598 18860 raft_consensus.cc:2781] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:55.659788 18860 raft_consensus.cc:686] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [term 1 LEADER]: Becoming Leader. State: Replica: 49a0cb2e3dad4a548936000ec3d80547, State: Running, Role: LEADER
I20260808 18:27:55.660158 18860 consensus_queue.cc:237] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } } peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } }
I20260808 18:27:55.663657 16985 catalog_manager.cc:5495] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49a0cb2e3dad4a548936000ec3d80547 (127.7.164.66). New cstate: current_term: 1 leader_uuid: "49a0cb2e3dad4a548936000ec3d80547" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } health_report { overall_health: HEALTHY } } }
W20260808 18:27:55.680917 18605 tablet.cc:2369] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:27:55.696898 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.693492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36704 (local address 127.7.164.66:37629)
0808 18:27:55.693671 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.693675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.693689 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:27:55.693767 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.693770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.693822 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.693911 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:55.694121 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.694696 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.695509 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.695723 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.695801 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.696074 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:55.696500 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:55.696608 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.696705 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.696761 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 18:27:55.703128 18332 raft_consensus.cc:1261] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe [term 1 FOLLOWER]: Refusing update from remote peer 49a0cb2e3dad4a548936000ec3d80547: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:55.703128 18727 raft_consensus.cc:1261] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030 [term 1 FOLLOWER]: Refusing update from remote peer 49a0cb2e3dad4a548936000ec3d80547: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:55.703903 18860 consensus_queue.cc:1035] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:55.704085 18863 consensus_queue.cc:1035] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:55.713116 18872 mvcc.cc:205] Tried to move back new op lower bound from 7316331215671926784 to 7316331215507013632. Current Snapshot: MvccSnapshot[applied={T|T < 7316331215671926784}]
I20260808 18:27:55.728179 16985 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:55.728358 16985 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:55.732857 16985 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0eb5242ae394100a33e0ced51baccb6: Sending DeleteTablet for 3 replicas of tablet 9272df1c4de344999296c99c443db182
I20260808 18:27:55.733561 18303 tablet_service.cc:1514] Processing DeleteTablet for tablet 9272df1c4de344999296c99c443db182 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:55 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:54042
I20260808 18:27:55.734696 18882 tablet_replica.cc:330] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe: stopping tablet replica
I20260808 18:27:55.734714 18702 tablet_service.cc:1514] Processing DeleteTablet for tablet 9272df1c4de344999296c99c443db182 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:55 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:46440
I20260808 18:27:55.735116 18882 raft_consensus.cc:2229] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:55.735311 18885 tablet_replica.cc:330] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030: stopping tablet replica
I20260808 18:27:55.735499 18882 raft_consensus.cc:2258] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:55.735651 18499 tablet_service.cc:1514] Processing DeleteTablet for tablet 9272df1c4de344999296c99c443db182 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:55 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:36700
I20260808 18:27:55.735831 18885 raft_consensus.cc:2229] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:55.736179 18885 raft_consensus.cc:2258] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:55.737219 18882 ts_tablet_manager.cc:1905] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:55.738521 18885 ts_tablet_manager.cc:1905] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:55.739936 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 16939
I20260808 18:27:55.748559 18882 ts_tablet_manager.cc:1918] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:27:55.748754 18882 log.cc:1196] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-0/wal/wals/9272df1c4de344999296c99c443db182
I20260808 18:27:55.749261 18882 ts_tablet_manager.cc:1939] T 9272df1c4de344999296c99c443db182 P b328e02ab41241ec9edc85a11d4da0fe: Deleting consensus metadata
I20260808 18:27:55.750362 18885 ts_tablet_manager.cc:1918] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:27:55.750561 18885 log.cc:1196] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/9272df1c4de344999296c99c443db182
I20260808 18:27:55.751132 18885 ts_tablet_manager.cc:1939] T 9272df1c4de344999296c99c443db182 P d3db78488e4f4f7489023ebb7002a030: Deleting consensus metadata
I20260808 18:27:55.754508 18886 tablet_replica.cc:330] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547: stopping tablet replica
I20260808 18:27:55.754951 18886 raft_consensus.cc:2229] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:27:55.755481 18886 raft_consensus.cc:2258] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:55.757557 18886 ts_tablet_manager.cc:1905] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 18:27:55.759240 18636 connection.cc:535] server connection from 127.0.0.1:46440 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 18:27:55.760054 18237 connection.cc:535] server connection from 127.0.0.1:54042 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 18:27:55.760689 18886 ts_tablet_manager.cc:1918] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:27:55.760879 18886 log.cc:1196] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/9272df1c4de344999296c99c443db182
I20260808 18:27:55.761294 18886 ts_tablet_manager.cc:1939] T 9272df1c4de344999296c99c443db182 P 49a0cb2e3dad4a548936000ec3d80547: Deleting consensus metadata
W20260808 18:27:55.762575 18435 connection.cc:441] server connection from 127.0.0.1:36700 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36700 (request call id 1) could send its response
I20260808 18:27:55.766161 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.739063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52748 (local address 127.7.164.125:43073)
0808 18:27:55.739200 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.739205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.754666 (+ 15461us) server_negotiation.cc:407] Connection header received
0808 18:27:55.754914 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.754920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.754992 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.755058 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:55.756359 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.757283 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.758798 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.759227 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.759320 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.761880 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:55.761887 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:55.761893 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:55.761928 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:55.763980 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.764539 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.764546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.764550 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.764658 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.765002 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.765005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.765007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.765259 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:55.765387 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.765818 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.765963 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:27:55.774600 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.764879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52762 (local address 127.7.164.125:43073)
0808 18:27:55.765014 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.765018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.765291 (+ 273us) server_negotiation.cc:407] Connection header received
0808 18:27:55.765469 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.765473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.765519 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.765569 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:55.766760 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.767565 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.768772 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.769115 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.769186 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.770945 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:55.770949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:55.770952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:55.770981 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:55.772677 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.773209 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.773212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.773214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.773299 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.773599 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.773602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.773604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.773816 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:55.773932 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.774316 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.774437 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:27:55.776400 18411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.765472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35428 (local address 127.7.164.124:38159)
0808 18:27:55.765599 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.765603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.766112 (+ 509us) server_negotiation.cc:407] Connection header received
0808 18:27:55.766333 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.766337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.766394 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.766449 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:55.767423 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.768166 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.769364 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.770002 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.770072 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.771765 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:55.771769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:55.771771 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:55.771792 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:55.773750 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.774634 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.774640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.774642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.774737 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.775345 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.775349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.775350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.775573 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:55.775699 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.776075 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.776203 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:27:55.777731 18607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:55.739202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35422 (local address 127.7.164.124:38159)
0808 18:27:55.739320 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:55.739385 (+ 65us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:55.767701 (+ 28316us) server_negotiation.cc:407] Connection header received
0808 18:27:55.767854 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:55.767857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:55.767903 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:55.767949 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:55.769157 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.769911 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.771133 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:55.771549 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:55.771633 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:55.773405 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:55.773409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:55.773412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:55.773440 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:55.775862 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.776329 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.776333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.776335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.776429 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:55.776754 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:55.776757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:55.776759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:55.776997 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:55.777092 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:55.777436 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 18:27:55.777559 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:27:56.716580 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:56.716008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.67:58301)
0808 18:27:56.716299 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:56.716427 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":85,"threads_started":1}
W20260808 18:27:56.716920 18807 heartbeater.cc:644] Failed to heartbeat to 127.7.164.126:35069 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
I20260808 18:27:56.720554 18905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:56.719873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.65:60479)
0808 18:27:56.720350 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:56.720447 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":71,"threads_started":1}
I20260808 18:27:56.720554 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:56.719578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.66:50541)
0808 18:27:56.720347 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:56.720447 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":94,"threads_started":1}
W20260808 18:27:56.720865 18603 heartbeater.cc:644] Failed to heartbeat to 127.7.164.126:35069 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
W20260808 18:27:56.720903 18399 heartbeater.cc:644] Failed to heartbeat to 127.7.164.126:35069 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
I20260808 18:27:57.233603 18907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b0eb5242ae394100a33e0ced51baccb6)
I20260808 18:27:57.233860 18907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } }
I20260808 18:27:57.234788 18908 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b0eb5242ae394100a33e0ced51baccb6)
I20260808 18:27:57.234972 18907 leader_election.cc:288] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b0eb5242ae394100a33e0ced51baccb6 (127.7.164.126:35069), 87f431983eab4f48bc40603dca5208f6 (127.7.164.124:38159)
I20260808 18:27:57.235493 18909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.235045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.0.0.1:44610)
0808 18:27:57.235344 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.235406 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":78,"threads_started":1}
I20260808 18:27:57.235431 18908 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } }
W20260808 18:27:57.235683 17831 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
I20260808 18:27:57.236517 18909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.236243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.0.0.1:44618)
0808 18:27:57.236400 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.236450 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260808 18:27:57.236743 17831 leader_election.cc:334] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b0eb5242ae394100a33e0ced51baccb6 (127.7.164.126:35069): Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
I20260808 18:27:57.236816 18908 leader_election.cc:288] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b0eb5242ae394100a33e0ced51baccb6 (127.7.164.126:35069), 49041fa054ed40ca9242a05b3d0eaddb (127.7.164.125:43073)
I20260808 18:27:57.237205 18914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.236503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.0.0.1:44634)
0808 18:27:57.237019 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.237110 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":91,"threads_started":1}
W20260808 18:27:57.237396 18108 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
I20260808 18:27:57.238329 18914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.238070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.0.0.1:44650)
0808 18:27:57.238190 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.238270 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 18:27:57.238504 18108 leader_election.cc:334] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b0eb5242ae394100a33e0ced51baccb6 (127.7.164.126:35069): Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
I20260808 18:27:57.239744 18910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.235186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:38159 (local address 127.0.0.1:35432)
0808 18:27:57.235582 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.235596 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:57.235692 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:57.236085 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:57.236088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:57.236105 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:57.236217 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:57.236223 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.237204 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.237211 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:57.238520 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:57.238530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.239449 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.239451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:57.239519 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.239540 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:27:57.239602 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 18:27:57.239654 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":78,"threads_started":1}
I20260808 18:27:57.239835 18911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.235358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35432 (local address 127.7.164.124:38159)
0808 18:27:57.235851 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:57.235857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:57.235876 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:27:57.235931 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:57.235934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:57.235984 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:57.236079 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:57.236339 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.237058 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.238686 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.239315 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.239452 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.239514 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:57.239640 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 18:27:57.239709 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":376,"thread_start_us":63,"threads_started":1}
I20260808 18:27:57.240218 18149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "49041fa054ed40ca9242a05b3d0eaddb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "87f431983eab4f48bc40603dca5208f6" is_pre_election: true
I20260808 18:27:57.240430 18149 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49041fa054ed40ca9242a05b3d0eaddb in term 1.
I20260808 18:27:57.240788 17835 leader_election.cc:302] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49041fa054ed40ca9242a05b3d0eaddb, 87f431983eab4f48bc40603dca5208f6; no voters: b0eb5242ae394100a33e0ced51baccb6
I20260808 18:27:57.241032 18907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 18:27:57.241098 18907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 FOLLOWER]: Starting leader election (detected failure of leader b0eb5242ae394100a33e0ced51baccb6)
I20260808 18:27:57.241148 18907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:27:57.241381 18918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.237281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52766 (local address 127.7.164.125:43073)
0808 18:27:57.237541 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:57.237550 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:57.237563 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:27:57.237604 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:57.237606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:57.237648 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:57.237730 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:57.238081 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.238810 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.240232 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.240809 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.240919 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.240972 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:57.241221 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 18:27:57.241277 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":96,"threads_started":1}
I20260808 18:27:57.241438 18917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.237003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:43073 (local address 127.0.0.1:52766)
0808 18:27:57.237201 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.237216 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:57.237391 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:57.237831 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:57.237834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:57.237852 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:57.237935 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:57.237952 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.238919 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.238923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:57.240119 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:57.240129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.240957 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.240963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:57.241074 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.241101 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:27:57.241193 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 18:27:57.241268 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":24,"thread_start_us":73,"threads_started":1}
I20260808 18:27:57.242347 18907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } }
I20260808 18:27:57.242733 18907 leader_election.cc:288] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [CANDIDATE]: Term 2 election: Requested vote from peers b0eb5242ae394100a33e0ced51baccb6 (127.7.164.126:35069), 87f431983eab4f48bc40603dca5208f6 (127.7.164.124:38159)
I20260808 18:27:57.242736 17858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87f431983eab4f48bc40603dca5208f6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "49041fa054ed40ca9242a05b3d0eaddb" is_pre_election: true
I20260808 18:27:57.243003 17858 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 87f431983eab4f48bc40603dca5208f6 in current term 2: Already voted for candidate 49041fa054ed40ca9242a05b3d0eaddb in this term.
I20260808 18:27:57.243058 18149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "49041fa054ed40ca9242a05b3d0eaddb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "87f431983eab4f48bc40603dca5208f6"
I20260808 18:27:57.243165 18149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:27:57.243407 18113 leader_election.cc:302] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87f431983eab4f48bc40603dca5208f6; no voters: 49041fa054ed40ca9242a05b3d0eaddb, b0eb5242ae394100a33e0ced51baccb6
I20260808 18:27:57.244257 18149 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49041fa054ed40ca9242a05b3d0eaddb in term 2.
I20260808 18:27:57.244407 18908 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:27:57.244639 18910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.242834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.0.0.1:44656)
0808 18:27:57.244518 (+ 1684us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.244576 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260808 18:27:57.244627 17835 leader_election.cc:302] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49041fa054ed40ca9242a05b3d0eaddb, 87f431983eab4f48bc40603dca5208f6; no voters:
I20260808 18:27:57.244827 18907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 2 FOLLOWER]: Leader election won for term 2
I20260808 18:27:57.245029 18907 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [term 2 LEADER]: Becoming Leader. State: Replica: 49041fa054ed40ca9242a05b3d0eaddb, State: Running, Role: LEADER
I20260808 18:27:57.245380 18910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.245155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.0.0.1:44658)
0808 18:27:57.245243 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.245293 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 18:27:57.245296 18907 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } }
W20260808 18:27:57.245479 17831 leader_election.cc:334] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b0eb5242ae394100a33e0ced51baccb6 (127.7.164.126:35069): Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
I20260808 18:27:57.246434 18920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49041fa054ed40ca9242a05b3d0eaddb. Latest consensus state: current_term: 2 leader_uuid: "49041fa054ed40ca9242a05b3d0eaddb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } } }
I20260808 18:27:57.246527 18920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:57.246809 18922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:57.248848 18922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:57.249195 18922 catalog_manager.cc:1259] Loaded cluster ID: 772ce27b4ff4457e9e891cd52b827ba7
I20260808 18:27:57.249238 18922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:57.249723 18922 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:57.250267 18922 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: Loaded TSK: 0
I20260808 18:27:57.250901 18922 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:27:57.290135 17851 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:27:57.293277 17851 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:27:57.296231 18910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.295966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.0.0.1:44682)
0808 18:27:57.296097 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.296162 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 18:27:57.296309 18149 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [term 2 FOLLOWER]: Refusing update from remote peer 49041fa054ed40ca9242a05b3d0eaddb: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 18:27:57.296880 18920 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [LEADER]: Connected to new peer: Peer: permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 18:27:57.297014 18910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.296751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.0.0.1:44698)
0808 18:27:57.296885 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.296953 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 18:27:57.297367 17831 consensus_peers.cc:487] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb -> Peer b0eb5242ae394100a33e0ced51baccb6 (127.7.164.126:35069): Couldn't send request to peer b0eb5242ae394100a33e0ced51baccb6. Status: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 18:27:57.298460 18908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49041fa054ed40ca9242a05b3d0eaddb. Latest consensus state: current_term: 2 leader_uuid: "49041fa054ed40ca9242a05b3d0eaddb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } } }
I20260808 18:27:57.298601 18908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:57.299746 18907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "49041fa054ed40ca9242a05b3d0eaddb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } } }
I20260808 18:27:57.299875 18907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:57.300314 18908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "49041fa054ed40ca9242a05b3d0eaddb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b0eb5242ae394100a33e0ced51baccb6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35069 } } peers { permanent_uuid: "49041fa054ed40ca9242a05b3d0eaddb" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 43073 } attrs { promote: false } } peers { permanent_uuid: "87f431983eab4f48bc40603dca5208f6" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 38159 } attrs { promote: false } } }
I20260808 18:27:57.300434 18908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f431983eab4f48bc40603dca5208f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:27:57.310122 18909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.306021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:37629 (local address 127.0.0.1:36710)
0808 18:27:57.306134 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.306149 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:57.306261 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:57.307131 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:57.307134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:57.307155 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:57.307252 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:57.307258 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.308133 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.308136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:57.308922 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:57.308931 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.309793 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.309796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:57.309895 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.309921 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 18:27:57.309981 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 18:27:57.310034 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":44}
I20260808 18:27:57.310446 18910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.305721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:42685 (local address 127.0.0.1:46452)
0808 18:27:57.305837 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.305854 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:57.305938 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:57.306705 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:57.306708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:57.306725 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:57.306823 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:57.306829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.307856 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.307859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:57.309044 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:57.309054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.310026 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.310029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:57.310116 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.310139 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:27:57.310207 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 18:27:57.310301 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":25}
I20260808 18:27:57.310681 18936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.306645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36710 (local address 127.7.164.66:37629)
0808 18:27:57.306872 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:57.306877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:57.306891 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:27:57.306940 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:57.306945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:57.307005 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:57.307359 (+ 354us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:57.307453 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.308022 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.309055 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.309662 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.310334 (+ 672us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.310399 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:57.310439 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 18:27:57.310538 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":76,"threads_started":1}
I20260808 18:27:57.310810 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.306266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:33189 (local address 127.0.0.1:54050)
0808 18:27:57.306521 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.306536 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:57.306622 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:57.307185 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:57.307188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:57.307207 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:57.307301 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:57.307307 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.308387 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.308390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:57.309537 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:57.309546 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.310470 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.310474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:57.310567 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.310593 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 18:27:57.310646 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 18:27:57.310704 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":4,"thread_start_us":106,"threads_started":1}
I20260808 18:27:57.310809 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.306073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46452 (local address 127.7.164.67:42685)
0808 18:27:57.306452 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:57.306459 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:57.306478 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:27:57.306539 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:57.306542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:57.306594 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:57.306695 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:57.306974 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.307729 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.309187 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.309802 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.309894 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.309969 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:57.310598 (+ 629us) server_negotiation.cc:299] Negotiation successful
0808 18:27:57.310679 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":128,"threads_started":1}
I20260808 18:27:57.311256 18499 tablet_service.cc:1467] Processing CreateTablet for tablet aee57fccfcc4460fbae9341d5137bf81 (DEFAULT_TABLE table=test-table [id=f6f6c3997fcf49c8b92e3e763515f9ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:57.311307 18702 tablet_service.cc:1467] Processing CreateTablet for tablet aee57fccfcc4460fbae9341d5137bf81 (DEFAULT_TABLE table=test-table [id=f6f6c3997fcf49c8b92e3e763515f9ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:57.311558 18499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aee57fccfcc4460fbae9341d5137bf81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:57.311583 18702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aee57fccfcc4460fbae9341d5137bf81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:57.313964 18937 tablet_bootstrap.cc:492] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547: Bootstrap starting.
I20260808 18:27:57.314040 18938 tablet_bootstrap.cc:492] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030: Bootstrap starting.
I20260808 18:27:57.314599 18935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.306378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54050 (local address 127.7.164.65:33189)
0808 18:27:57.306941 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:57.306946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:57.306959 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:27:57.307010 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:57.307015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:57.307062 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:57.307405 (+ 343us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:57.307528 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.308256 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.309685 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.310328 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.310999 (+ 671us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.311061 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:57.311097 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 18:27:57.314324 (+ 3227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":97,"threads_started":1}
I20260808 18:27:57.315064 18937 tablet_bootstrap.cc:654] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:57.315143 18938 tablet_bootstrap.cc:654] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:57.315119 18303 tablet_service.cc:1467] Processing CreateTablet for tablet aee57fccfcc4460fbae9341d5137bf81 (DEFAULT_TABLE table=test-table [id=f6f6c3997fcf49c8b92e3e763515f9ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:27:57.315400 18303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aee57fccfcc4460fbae9341d5137bf81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:57.316255 18937 tablet_bootstrap.cc:492] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547: No bootstrap required, opened a new log
I20260808 18:27:57.316363 18937 ts_tablet_manager.cc:1397] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:57.316818 18937 raft_consensus.cc:348] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } } peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } }
I20260808 18:27:57.316941 18937 raft_consensus.cc:374] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:57.316982 18937 raft_consensus.cc:729] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a0cb2e3dad4a548936000ec3d80547, State: Initialized, Role: FOLLOWER
I20260808 18:27:57.317098 18937 consensus_queue.cc:260] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } } peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } }
I20260808 18:27:57.317453 18937 ts_tablet_manager.cc:1428] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:57.318012 18946 tablet_bootstrap.cc:492] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe: Bootstrap starting.
I20260808 18:27:57.319031 18946 tablet_bootstrap.cc:654] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:57.319125 18938 tablet_bootstrap.cc:492] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030: No bootstrap required, opened a new log
I20260808 18:27:57.319204 18938 ts_tablet_manager.cc:1397] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 18:27:57.319662 18938 raft_consensus.cc:348] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } } peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } }
I20260808 18:27:57.319816 18938 raft_consensus.cc:374] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:57.319871 18938 raft_consensus.cc:729] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3db78488e4f4f7489023ebb7002a030, State: Initialized, Role: FOLLOWER
I20260808 18:27:57.320056 18604 heartbeater.cc:503] Master 127.7.164.125:43073 was elected leader, sending a full tablet report...
I20260808 18:27:57.320030 18938 consensus_queue.cc:260] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } } peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } }
I20260808 18:27:57.320302 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.317776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.66:46965)
0808 18:27:57.319990 (+ 2214us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.320055 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2152,"thread_start_us":92,"threads_started":1}
I20260808 18:27:57.320380 18938 ts_tablet_manager.cc:1428] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:27:57.321110 18946 tablet_bootstrap.cc:492] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe: No bootstrap required, opened a new log
I20260808 18:27:57.321208 18946 ts_tablet_manager.cc:1397] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:27:57.321103 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.320834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.66:55679)
0808 18:27:57.320985 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.321038 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 18:27:57.322429 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.321977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.67:58963)
0808 18:27:57.322266 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.322324 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":121,"threads_started":1}
I20260808 18:27:57.322403 18946 raft_consensus.cc:348] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } } peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } }
I20260808 18:27:57.322574 18946 raft_consensus.cc:374] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:57.322623 18946 raft_consensus.cc:729] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b328e02ab41241ec9edc85a11d4da0fe, State: Initialized, Role: FOLLOWER
I20260808 18:27:57.322789 18946 consensus_queue.cc:260] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } } peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } }
I20260808 18:27:57.323173 18946 ts_tablet_manager.cc:1428] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 18:27:57.323323 18808 heartbeater.cc:503] Master 127.7.164.125:43073 was elected leader, sending a full tablet report...
I20260808 18:27:57.323985 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.323760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.67:54621)
0808 18:27:57.323871 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.323927 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 18:27:57.324301 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.323570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.65:45723)
0808 18:27:57.324125 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.324188 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":119,"threads_started":1}
I20260808 18:27:57.325446 18400 heartbeater.cc:503] Master 127.7.164.125:43073 was elected leader, sending a full tablet report...
I20260808 18:27:57.325745 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.325470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.65:49869)
0808 18:27:57.325632 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.325681 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 18:27:57.384176 18951 raft_consensus.cc:482] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:27:57.384336 18951 raft_consensus.cc:504] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } } peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } }
I20260808 18:27:57.385449 18951 leader_election.cc:288] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49a0cb2e3dad4a548936000ec3d80547 (127.7.164.66:37629), b328e02ab41241ec9edc85a11d4da0fe (127.7.164.65:33189)
I20260808 18:27:57.390825 18936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.385878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:43277 (local address 127.7.164.66:37629)
0808 18:27:57.386035 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:57.386039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:57.386056 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:27:57.386114 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:57.386119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:57.386171 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:57.386287 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:57.386685 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.387507 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.389489 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.390095 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.390243 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.390299 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:57.390619 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 18:27:57.390693 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 18:27:57.390825 18935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.386168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:39587 (local address 127.7.164.65:33189)
0808 18:27:57.386334 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:57.386338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:57.386504 (+ 166us) server_negotiation.cc:407] Connection header received
0808 18:27:57.386587 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:57.386589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:57.386654 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:57.386735 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:57.386977 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.387689 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.389489 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.390093 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.390242 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.390299 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:57.390619 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 18:27:57.390693 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 18:27:57.390892 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.385725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:37629 (local address 127.7.164.67:43277)
0808 18:27:57.385882 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.385895 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:57.386021 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:57.386385 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:57.386389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:57.386412 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:57.386561 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:57.386569 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.387643 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.387646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:57.389342 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:57.389355 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.390287 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.390292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:57.390489 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.390509 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 18:27:57.390591 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 18:27:57.390705 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":135}
I20260808 18:27:57.391139 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.386036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:33189 (local address 127.7.164.67:39587)
0808 18:27:57.386396 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.386409 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:57.386504 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:57.386761 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:57.386763 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:57.386779 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:57.386867 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:57.386871 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.387810 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.387814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:57.389342 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:57.389355 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.390287 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.390292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:57.390459 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.390495 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 18:27:57.390591 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 18:27:57.390705 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":176,"thread_start_us":133,"threads_started":1}
I20260808 18:27:57.391528 18527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aee57fccfcc4460fbae9341d5137bf81" candidate_uuid: "d3db78488e4f4f7489023ebb7002a030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a0cb2e3dad4a548936000ec3d80547" is_pre_election: true
I20260808 18:27:57.391593 18332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aee57fccfcc4460fbae9341d5137bf81" candidate_uuid: "d3db78488e4f4f7489023ebb7002a030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b328e02ab41241ec9edc85a11d4da0fe" is_pre_election: true
I20260808 18:27:57.391758 18332 raft_consensus.cc:2454] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3db78488e4f4f7489023ebb7002a030 in term 0.
I20260808 18:27:57.391821 18527 raft_consensus.cc:2454] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3db78488e4f4f7489023ebb7002a030 in term 0.
I20260808 18:27:57.392287 18637 leader_election.cc:302] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49a0cb2e3dad4a548936000ec3d80547, d3db78488e4f4f7489023ebb7002a030; no voters:
I20260808 18:27:57.392596 18951 raft_consensus.cc:2781] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:27:57.392685 18951 raft_consensus.cc:482] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:27:57.392740 18951 raft_consensus.cc:3037] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:57.394001 18951 raft_consensus.cc:504] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } } peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } }
I20260808 18:27:57.394410 18951 leader_election.cc:288] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [CANDIDATE]: Term 1 election: Requested vote from peers 49a0cb2e3dad4a548936000ec3d80547 (127.7.164.66:37629), b328e02ab41241ec9edc85a11d4da0fe (127.7.164.65:33189)
I20260808 18:27:57.394799 18527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aee57fccfcc4460fbae9341d5137bf81" candidate_uuid: "d3db78488e4f4f7489023ebb7002a030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a0cb2e3dad4a548936000ec3d80547"
I20260808 18:27:57.394843 18332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aee57fccfcc4460fbae9341d5137bf81" candidate_uuid: "d3db78488e4f4f7489023ebb7002a030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b328e02ab41241ec9edc85a11d4da0fe"
I20260808 18:27:57.394908 18527 raft_consensus.cc:3037] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:57.394976 18332 raft_consensus.cc:3037] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:57.395892 18332 raft_consensus.cc:2454] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3db78488e4f4f7489023ebb7002a030 in term 1.
I20260808 18:27:57.395980 18527 raft_consensus.cc:2454] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3db78488e4f4f7489023ebb7002a030 in term 1.
I20260808 18:27:57.396175 18637 leader_election.cc:302] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b328e02ab41241ec9edc85a11d4da0fe, d3db78488e4f4f7489023ebb7002a030; no voters:
I20260808 18:27:57.396330 18951 raft_consensus.cc:2781] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:57.396570 18951 raft_consensus.cc:686] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [term 1 LEADER]: Becoming Leader. State: Replica: d3db78488e4f4f7489023ebb7002a030, State: Running, Role: LEADER
I20260808 18:27:57.396883 18951 consensus_queue.cc:237] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } } peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } }
I20260808 18:27:57.397284 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.397051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.67:36197)
0808 18:27:57.397154 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.397218 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 18:27:57.399253 17851 catalog_manager.cc:5495] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3db78488e4f4f7489023ebb7002a030 (127.7.164.67). New cstate: current_term: 1 leader_uuid: "d3db78488e4f4f7489023ebb7002a030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3db78488e4f4f7489023ebb7002a030" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 42685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20260808 18:27:57.433391 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.429648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46460 (local address 127.7.164.67:42685)
0808 18:27:57.429812 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:57.429817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:57.429833 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:27:57.429934 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:57.429937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:57.429995 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:57.430093 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:27:57.430422 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.430944 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.431837 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:57.432121 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:57.432207 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:57.432506 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:27:57.432995 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:27:57.433120 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:57.433164 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 18:27:57.433217 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 18:27:57.436332 18332 raft_consensus.cc:1261] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe [term 1 FOLLOWER]: Refusing update from remote peer d3db78488e4f4f7489023ebb7002a030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:57.436554 18527 raft_consensus.cc:1261] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547 [term 1 FOLLOWER]: Refusing update from remote peer d3db78488e4f4f7489023ebb7002a030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:27:57.436939 18959 consensus_queue.cc:1035] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b328e02ab41241ec9edc85a11d4da0fe" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:57.437181 18951 consensus_queue.cc:1035] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49a0cb2e3dad4a548936000ec3d80547" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 37629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:27:57.438936 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.438608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.65:55045)
0808 18:27:57.438765 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.438836 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 18:27:57.439950 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.439655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.66:50683)
0808 18:27:57.439806 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.439859 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 18:27:57.442683 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.442412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.67:52017)
0808 18:27:57.442561 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.442613 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260808 18:27:57.444983 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.444706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.66:54099)
0808 18:27:57.444862 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.444913 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 18:27:57.448408 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.448138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.65:36977)
0808 18:27:57.448270 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.448330 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 18:27:57.448680 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.446712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35069 (local address 127.7.164.67:55753)
0808 18:27:57.448556 (+ 1844us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.448611 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1772}
I20260808 18:27:57.450814 17847 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:57.450984 17847 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:27:57.454838 17847 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49041fa054ed40ca9242a05b3d0eaddb: Sending DeleteTablet for 3 replicas of tablet aee57fccfcc4460fbae9341d5137bf81
I20260808 18:27:57.455549 18702 tablet_service.cc:1514] Processing DeleteTablet for tablet aee57fccfcc4460fbae9341d5137bf81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:57 UTC) from {username='oryx', principal='oryx/127.7.164.125@KRBTEST.COM'} at 127.0.0.1:46452
I20260808 18:27:57.455669 18303 tablet_service.cc:1514] Processing DeleteTablet for tablet aee57fccfcc4460fbae9341d5137bf81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:57 UTC) from {username='oryx', principal='oryx/127.7.164.125@KRBTEST.COM'} at 127.0.0.1:54050
I20260808 18:27:57.456190 18977 tablet_replica.cc:330] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe: stopping tablet replica
I20260808 18:27:57.456187 18499 tablet_service.cc:1514] Processing DeleteTablet for tablet aee57fccfcc4460fbae9341d5137bf81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:27:57 UTC) from {username='oryx', principal='oryx/127.7.164.125@KRBTEST.COM'} at 127.0.0.1:36710
I20260808 18:27:57.456341 18977 raft_consensus.cc:2229] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:57.456482 18977 raft_consensus.cc:2258] T aee57fccfcc4460fbae9341d5137bf81 P b328e02ab41241ec9edc85a11d4da0fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:57.456519 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 18072
I20260808 18:27:57.466487 18978 tablet_replica.cc:330] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547: stopping tablet replica
I20260808 18:27:57.466495 18976 tablet_replica.cc:330] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030: stopping tablet replica
I20260808 18:27:57.466658 18978 raft_consensus.cc:2229] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:27:57.466665 18976 raft_consensus.cc:2229] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:27:57.466811 18978 raft_consensus.cc:2258] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:57.467398 18976 raft_consensus.cc:2258] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:27:57.467810 18978 ts_tablet_manager.cc:1905] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:57.468223 18976 ts_tablet_manager.cc:1905] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:27:57.470984 18978 ts_tablet_manager.cc:1918] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:27:57.471084 18978 log.cc:1196] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/aee57fccfcc4460fbae9341d5137bf81
I20260808 18:27:57.471392 18978 ts_tablet_manager.cc:1939] T aee57fccfcc4460fbae9341d5137bf81 P 49a0cb2e3dad4a548936000ec3d80547: Deleting consensus metadata
I20260808 18:27:57.471683 18976 ts_tablet_manager.cc:1918] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:27:57.471773 18976 log.cc:1196] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/aee57fccfcc4460fbae9341d5137bf81
I20260808 18:27:57.472067 18976 ts_tablet_manager.cc:1939] T aee57fccfcc4460fbae9341d5137bf81 P d3db78488e4f4f7489023ebb7002a030: Deleting consensus metadata
I20260808 18:27:57.472422 17837 catalog_manager.cc:4841] TS 49a0cb2e3dad4a548936000ec3d80547 (127.7.164.66:37629): tablet aee57fccfcc4460fbae9341d5137bf81 (table test-table [id=f6f6c3997fcf49c8b92e3e763515f9ba]) successfully deleted
I20260808 18:27:57.472900 17834 catalog_manager.cc:4841] TS d3db78488e4f4f7489023ebb7002a030 (127.7.164.67:42685): tablet aee57fccfcc4460fbae9341d5137bf81 (table test-table [id=f6f6c3997fcf49c8b92e3e763515f9ba]) successfully deleted
I20260808 18:27:57.475145 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 18412
I20260808 18:27:57.475335 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:57.475055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:33189 (local address 127.0.0.1:54066)
0808 18:27:57.475172 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:57.475234 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:33189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 18:27:57.479466 17837 catalog_manager.cc:4584] TS b328e02ab41241ec9edc85a11d4da0fe (127.7.164.65:33189): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aee57fccfcc4460fbae9341d5137bf81: Network error: Client connection negotiation failed: client connection to 127.7.164.65:33189: connect: Connection refused (error 111)
I20260808 18:27:57.490831 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 18610
I20260808 18:27:57.504179 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 17652
I20260808 18:27:57.516610 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 17799
2026-08-08T18:27:57Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5757 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:27:57 dist-test-slave-jztv krb5kdc[18983](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:27:57 dist-test-slave-jztv krb5kdc[18983](info): set up 2 sockets
Aug 08 18:27:57 dist-test-slave-jztv krb5kdc[18983](info): commencing operation
krb5kdc: starting...
W20260808 18:27:58.738904 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:27:58 dist-test-slave-jztv krb5kdc[18983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:27:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:27:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.126.keytab.
Entry for principal oryx/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.126.keytab.
I20260808 18:27:58.873490 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:32975
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.126.keytab
--principal=oryx/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:35091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:32975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:58.988719 19002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:58.989079 19002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:58.989161 19002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:58.993261 19002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:27:58.993366 19002 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 18:27:58.993407 19002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:58.993448 19002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:58.993489 19002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:27:58.993533 19002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:27:58.999066 19002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:35091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:32975
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.126.keytab
--principal=oryx/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:32975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:59.000306 19002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:59.001389 19002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:59.010061 19011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:59.010051 19009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:59.010072 19013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:59.010342 19002 server_base.cc:1034] running on GCE node
I20260808 18:27:59.011027 19002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:59.012079 19002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:59.013249 19002 hybrid_clock.cc:648] HybridClock initialized: now 1786213679013216 us; error 42 us; skew 500 ppm
Aug 08 18:27:59 dist-test-slave-jztv krb5kdc[18983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213679, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:59.019095 19002 init.cc:377] Logged in from keytab as oryx/127.7.164.126@KRBTEST.COM (short username oryx)
I20260808 18:27:59.020063 19002 webserver.cc:460] Webserver started at http://127.7.164.126:45545/ using document root <none> and password file <none>
I20260808 18:27:59.020721 19002 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:59.020816 19002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:59.021061 19002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:59.023211 19002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "92a8443796184dafa26ae109ebb627a9"
format_stamp: "Formatted at 2026-08-08 18:27:59 on dist-test-slave-jztv"
server_key: "11ad8a816a9fdf25a55b974ad6d4f9df"
server_key_iv: "a557382457db6ff3869c76faf3710125"
server_key_version: "encryptionkey@0"
I20260808 18:27:59.023818 19002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "92a8443796184dafa26ae109ebb627a9"
format_stamp: "Formatted at 2026-08-08 18:27:59 on dist-test-slave-jztv"
server_key: "11ad8a816a9fdf25a55b974ad6d4f9df"
server_key_iv: "a557382457db6ff3869c76faf3710125"
server_key_version: "encryptionkey@0"
I20260808 18:27:59.027527 19002 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:27:59.029861 19025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:59.031165 19002 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:27:59.031307 19002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "92a8443796184dafa26ae109ebb627a9"
format_stamp: "Formatted at 2026-08-08 18:27:59 on dist-test-slave-jztv"
server_key: "11ad8a816a9fdf25a55b974ad6d4f9df"
server_key_iv: "a557382457db6ff3869c76faf3710125"
server_key_version: "encryptionkey@0"
I20260808 18:27:59.031424 19002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:59.054688 19002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:59.058048 19002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:59.058328 19002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:59.066506 19002 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:32975
I20260808 18:27:59.066529 19105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:32975 every 8 connection(s)
I20260808 18:27:59.067550 19002 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:27:59.070341 19106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:27:59.071368 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 19002
I20260808 18:27:59.071501 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:27:59.071777 7825 external_mini_cluster.cc:1428] Setting key 3b87a0ab40b5f50f8f71bd60fcfed3f5
I20260808 18:27:59.076282 19106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9: Bootstrap starting.
I20260808 18:27:59.078511 19106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9: Neither blocks nor log segments found. Creating new log.
I20260808 18:27:59.079233 19106 log.cc:824] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:27:59 dist-test-slave-jztv krb5kdc[18983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
I20260808 18:27:59.081318 19106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9: No bootstrap required, opened a new log
I20260808 18:27:59.084632 19106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a8443796184dafa26ae109ebb627a9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32975 } }
I20260808 18:27:59.084925 19106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:27:59.085024 19106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92a8443796184dafa26ae109ebb627a9, State: Initialized, Role: FOLLOWER
I20260808 18:27:59.085474 19106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a8443796184dafa26ae109ebb627a9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32975 } }
I20260808 18:27:59.085626 19106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:27:59.085716 19106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:27:59.085834 19106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:27:59.085973 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.073178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.7.164.126:32975)
0808 18:27:59.073596 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:59.073611 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:59.073644 (+ 33us) server_negotiation.cc:407] Connection header received
0808 18:27:59.074342 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:59.074361 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:59.074678 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:59.075010 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:59.075825 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.076829 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.077668 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.078148 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.078362 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.080929 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:59.080944 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:59.080960 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:59.080993 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:59.082830 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.083311 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.083317 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.083321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.083431 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.083734 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.083738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.083740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.084133 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:59.084344 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:59.084658 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 18:27:59.084985 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":134,"threads_started":1}
I20260808 18:27:59.086824 19106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a8443796184dafa26ae109ebb627a9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32975 } }
I20260808 18:27:59.087177 19106 leader_election.cc:302] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92a8443796184dafa26ae109ebb627a9; no voters:
I20260808 18:27:59.087505 19106 leader_election.cc:288] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:27:59.087610 19115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:27:59.087831 19115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [term 1 LEADER]: Becoming Leader. State: Replica: 92a8443796184dafa26ae109ebb627a9, State: Running, Role: LEADER
I20260808 18:27:59.088150 19115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a8443796184dafa26ae109ebb627a9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32975 } }
I20260808 18:27:59.088769 19106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:27:59.089823 19117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92a8443796184dafa26ae109ebb627a9. Latest consensus state: current_term: 1 leader_uuid: "92a8443796184dafa26ae109ebb627a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a8443796184dafa26ae109ebb627a9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32975 } } }
I20260808 18:27:59.089973 19117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:59.090283 19116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92a8443796184dafa26ae109ebb627a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a8443796184dafa26ae109ebb627a9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32975 } } }
I20260808 18:27:59.090379 19116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:27:59.090494 19126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:27:59.093593 19126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:27:59.099339 19126 catalog_manager.cc:1347] Generated new cluster ID: d227ac771adc4c47a2bd4ba3f9c37d7e
I20260808 18:27:59.099408 19126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:27:59.118251 19126 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:27:59.119254 19126 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:27:59.128558 19126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9: Generated new TSK 0
I20260808 18:27:59.129244 19126 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.65.keytab.
Entry for principal oryx/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.65.keytab.
I20260808 18:27:59.170310 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:32975
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.65.keytab
--principal=oryx/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:35091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:59.337296 19142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:59.337775 19142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:59.337824 19142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:59.344069 19142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:59.344226 19142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:59.344302 19142 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:27:59.344367 19142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:27:59.353129 19142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:35091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.65.keytab
--principal=oryx/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.164.126:32975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:59.355820 19142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:59.357842 19142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:59.374356 19152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:59.374708 19142 server_base.cc:1034] running on GCE node
W20260808 18:27:59.374356 19157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:59.374372 19153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:59.375581 19142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:59.376607 19142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:59.377851 19142 hybrid_clock.cc:648] HybridClock initialized: now 1786213679377809 us; error 62 us; skew 500 ppm
Aug 08 18:27:59 dist-test-slave-jztv krb5kdc[18983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213679, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:59.390918 19142 init.cc:377] Logged in from keytab as oryx/127.7.164.65@KRBTEST.COM (short username oryx)
I20260808 18:27:59.392693 19142 webserver.cc:460] Webserver started at http://127.7.164.65:46217/ using document root <none> and password file <none>
I20260808 18:27:59.393702 19142 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:59.393795 19142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:59.394155 19142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:59.397784 19142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "892d0db548c4449c8d67d8f629d38257"
format_stamp: "Formatted at 2026-08-08 18:27:59 on dist-test-slave-jztv"
server_key: "7f757b723d04b6a0254c1ead7e3a4db7"
server_key_iv: "5d475c8036559896d4966ebd041fa47e"
server_key_version: "encryptionkey@0"
I20260808 18:27:59.398878 19142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "892d0db548c4449c8d67d8f629d38257"
format_stamp: "Formatted at 2026-08-08 18:27:59 on dist-test-slave-jztv"
server_key: "7f757b723d04b6a0254c1ead7e3a4db7"
server_key_iv: "5d475c8036559896d4966ebd041fa47e"
server_key_version: "encryptionkey@0"
I20260808 18:27:59.405722 19142 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 18:27:59.410346 19165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:59.412101 19142 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260808 18:27:59.412250 19142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "892d0db548c4449c8d67d8f629d38257"
format_stamp: "Formatted at 2026-08-08 18:27:59 on dist-test-slave-jztv"
server_key: "7f757b723d04b6a0254c1ead7e3a4db7"
server_key_iv: "5d475c8036559896d4966ebd041fa47e"
server_key_version: "encryptionkey@0"
I20260808 18:27:59.412375 19142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:59.433274 19142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:59.436595 19142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:59.436810 19142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:59.438364 19142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:59.438443 19142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:59.438519 19142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:59.438562 19142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:27:59 dist-test-slave-jztv krb5kdc[18983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213679, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
I20260808 18:27:59.450309 19142 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:33565
I20260808 18:27:59.450371 19328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:33565 every 8 connection(s)
I20260808 18:27:59.451455 19142 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:27:59.452665 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.439239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:33133 (local address 127.7.164.126:32975)
0808 18:27:59.439381 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:59.439384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:59.440482 (+ 1098us) server_negotiation.cc:407] Connection header received
0808 18:27:59.441487 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:59.441491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:59.441551 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:59.441613 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:59.443474 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.444129 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.445296 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.445539 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.445652 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.448577 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:59.448581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:59.448583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:59.448608 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:59.450331 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.450928 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.450931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.450932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.451021 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.451394 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.451397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.451398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.451632 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:59.451712 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:59.452352 (+ 640us) server_negotiation.cc:299] Negotiation successful
0808 18:27:59.452492 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:27:59.453676 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.439808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:32975 (local address 127.7.164.65:33133)
0808 18:27:59.440337 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:59.440380 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:59.441223 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:59.442097 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:59.442108 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:59.442654 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:59.443231 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.443259 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.444270 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.444274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.445155 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.445165 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.445725 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.445730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.445955 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.446636 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:59.446663 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:59.448385 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:59.450475 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:59.450481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:59.450495 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:59.450819 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:59.451124 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:59.451127 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:59.451129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:59.451296 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:59.451737 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:59.451747 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:59.452083 (+ 336us) client_negotiation.cc:769] Sending connection context
0808 18:27:59.452303 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 18:27:59.452614 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":132,"threads_started":1}
I20260808 18:27:59.454994 19329 heartbeater.cc:347] Connected to a master server at 127.7.164.126:32975
I20260808 18:27:59.455165 19329 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:59.455710 19329 heartbeater.cc:511] Master 127.7.164.126:32975 requested a full tablet report, sending...
I20260808 18:27:59.457332 19045 ts_manager.cc:194] Registered new tserver with Master: 892d0db548c4449c8d67d8f629d38257 (127.7.164.65:33565)
I20260808 18:27:59.458535 19045 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:33133
I20260808 18:27:59.460631 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 19142
I20260808 18:27:59.460767 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:27:59.461117 7825 external_mini_cluster.cc:1428] Setting key 555f5158172e9c8a0f6634875410679d
I20260808 18:27:59.466670 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.457105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:44201 (local address 127.7.164.126:32975)
0808 18:27:59.457226 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:59.457230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:59.459296 (+ 2066us) server_negotiation.cc:407] Connection header received
0808 18:27:59.459450 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:59.459454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:59.459501 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:59.459588 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:59.459994 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.460723 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.461795 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.462084 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.462193 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.463448 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:59.463451 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:59.463453 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:59.463476 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:59.464899 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.465431 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.465434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.465435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.465525 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.465845 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.465848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.465850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.466019 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:59.466099 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:59.466450 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 18:27:59.466549 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":51}
I20260808 18:27:59.466873 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.457046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:32975 (local address 127.7.164.65:44201)
0808 18:27:59.459199 (+ 2153us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:59.459215 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:59.459318 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:59.459611 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:59.459613 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:59.459784 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:59.459881 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.459886 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.460907 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.460912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.461667 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.461677 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.462209 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.462236 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.462337 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.462798 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:59.462812 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:59.463288 (+ 476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:59.465014 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:59.465019 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:59.465023 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:59.465304 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:59.465631 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:59.465636 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:59.465639 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:59.465749 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:59.466125 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:59.466128 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:59.466278 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 18:27:59.466400 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 18:27:59.466585 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2067,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for oryx/127.7.164.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.164.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.66.keytab.
Entry for principal oryx/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.66.keytab.
I20260808 18:27:59.491354 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:32975
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.66.keytab
--principal=oryx/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:35091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:59.605845 19342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:59.606179 19342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:59.606307 19342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:59.610532 19342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:59.610615 19342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:59.610680 19342 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:27:59.610764 19342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:27:59.616133 19342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:35091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.66.keytab
--principal=oryx/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.164.126:32975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:59.617259 19342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:59.618389 19342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:59.628461 19351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:59.628469 19354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:59.628439 19349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:59.628717 19342 server_base.cc:1034] running on GCE node
I20260808 18:27:59.629238 19342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:59.629963 19342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:59.631143 19342 hybrid_clock.cc:648] HybridClock initialized: now 1786213679631116 us; error 37 us; skew 500 ppm
Aug 08 18:27:59 dist-test-slave-jztv krb5kdc[18983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213679, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:59.633889 19342 init.cc:377] Logged in from keytab as oryx/127.7.164.66@KRBTEST.COM (short username oryx)
I20260808 18:27:59.634845 19342 webserver.cc:460] Webserver started at http://127.7.164.66:33983/ using document root <none> and password file <none>
I20260808 18:27:59.635416 19342 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:59.635479 19342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:59.635658 19342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:59.638157 19342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "906fd6cd59e44e3db022089c7c1b14b5"
format_stamp: "Formatted at 2026-08-08 18:27:59 on dist-test-slave-jztv"
server_key: "3be5ed5986fc6b998010cbd1f3f90457"
server_key_iv: "00ddc1b04de129add1ae862fb30798e2"
server_key_version: "encryptionkey@0"
I20260808 18:27:59.638773 19342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "906fd6cd59e44e3db022089c7c1b14b5"
format_stamp: "Formatted at 2026-08-08 18:27:59 on dist-test-slave-jztv"
server_key: "3be5ed5986fc6b998010cbd1f3f90457"
server_key_iv: "00ddc1b04de129add1ae862fb30798e2"
server_key_version: "encryptionkey@0"
I20260808 18:27:59.642656 19342 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:27:59.645169 19363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:59.646495 19342 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:59.646631 19342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "906fd6cd59e44e3db022089c7c1b14b5"
format_stamp: "Formatted at 2026-08-08 18:27:59 on dist-test-slave-jztv"
server_key: "3be5ed5986fc6b998010cbd1f3f90457"
server_key_iv: "00ddc1b04de129add1ae862fb30798e2"
server_key_version: "encryptionkey@0"
I20260808 18:27:59.646746 19342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:59.686847 19342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:59.690472 19342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:59.690716 19342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:59.692276 19342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:59.692355 19342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:59.692432 19342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:59.692476 19342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:27:59 dist-test-slave-jztv krb5kdc[18983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213679, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
I20260808 18:27:59.704191 19342 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:38205
I20260808 18:27:59.704336 19527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:38205 every 8 connection(s)
I20260808 18:27:59.705418 19342 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:27:59.707592 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.693234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:52933 (local address 127.7.164.126:32975)
0808 18:27:59.693368 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:59.693371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:59.694521 (+ 1150us) server_negotiation.cc:407] Connection header received
0808 18:27:59.695570 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:59.695576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:59.695648 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:59.695715 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:59.697606 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.698530 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.700055 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.700288 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.700406 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.703270 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:59.703274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:59.703277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:59.703310 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:59.705172 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.705802 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.705808 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.705812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.705918 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.706377 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.706384 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.706385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.706631 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:59.706760 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:59.707281 (+ 521us) server_negotiation.cc:299] Negotiation successful
0808 18:27:59.707413 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:27:59.708364 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.693816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:32975 (local address 127.7.164.66:52933)
0808 18:27:59.694367 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:59.694414 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:59.695285 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:59.696219 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:59.696229 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:59.696719 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:59.697443 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.697465 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.698676 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.698680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.699910 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.699921 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.700475 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.700480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.700667 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.701317 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:59.701336 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:59.703078 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:59.705327 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:59.705332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:59.705345 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:59.705692 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:59.706038 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:59.706042 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:59.706044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:59.706257 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:59.706757 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:59.706763 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:59.707020 (+ 257us) client_negotiation.cc:769] Sending connection context
0808 18:27:59.707217 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 18:27:59.707558 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":153,"threads_started":1}
I20260808 18:27:59.708995 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 19342
I20260808 18:27:59.709103 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:27:59.709444 7825 external_mini_cluster.cc:1428] Setting key 11cfc773acd641b3aa3ae1fbd9d32e7d
I20260808 18:27:59.709800 19528 heartbeater.cc:347] Connected to a master server at 127.7.164.126:32975
I20260808 18:27:59.709966 19528 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:59.710549 19528 heartbeater.cc:511] Master 127.7.164.126:32975 requested a full tablet report, sending...
I20260808 18:27:59.711573 19045 ts_manager.cc:194] Registered new tserver with Master: 906fd6cd59e44e3db022089c7c1b14b5 (127.7.164.66:38205)
I20260808 18:27:59.712296 19045 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:52933
I20260808 18:27:59.720235 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.712607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:51947 (local address 127.7.164.126:32975)
0808 18:27:59.712742 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:59.712746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:59.712760 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:27:59.712795 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:59.712798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:59.712839 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:59.712927 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:59.713521 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.714374 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.715210 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.715569 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.715672 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.716926 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:59.716930 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:59.716933 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:59.716958 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:59.718513 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.718947 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.718951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.718953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.719045 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.719359 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.719364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.719366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.719585 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:59.719699 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:59.719965 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 18:27:59.720085 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:27:59.720248 19535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.712019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:32975 (local address 127.7.164.66:51947)
0808 18:27:59.712330 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:59.712346 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:59.712475 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:59.712965 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:59.712969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:59.713213 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:59.713384 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.713393 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.714528 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.714531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.715084 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.715092 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.715681 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.715684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.715781 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.716353 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:59.716366 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:59.716793 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:59.718628 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:59.718633 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:59.718635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:59.718843 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:59.719149 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:59.719152 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:59.719154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:59.719251 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:59.719690 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:59.719693 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:59.719796 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 18:27:59.719907 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 18:27:59.720059 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":4,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for oryx/127.7.164.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.164.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.67.keytab.
Entry for principal oryx/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.67.keytab.
I20260808 18:27:59.741904 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:32975
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.67.keytab
--principal=oryx/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:35091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:27:59.861922 19540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:27:59.862196 19540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:27:59.862262 19540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:27:59.866134 19540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:27:59.866184 19540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:27:59.866241 19540 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:27:59.866274 19540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:27:59.871300 19540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:35091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.67.keytab
--principal=oryx/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.164.126:32975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:27:59.872262 19540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:27:59.873348 19540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:27:59.883111 19548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:59.883189 19549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:27:59.883111 19551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:27:59.883325 19540 server_base.cc:1034] running on GCE node
I20260808 18:27:59.883780 19540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:27:59.884421 19540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:27:59.885624 19540 hybrid_clock.cc:648] HybridClock initialized: now 1786213679885609 us; error 27 us; skew 500 ppm
Aug 08 18:27:59 dist-test-slave-jztv krb5kdc[18983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213679, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:27:59.888463 19540 init.cc:377] Logged in from keytab as oryx/127.7.164.67@KRBTEST.COM (short username oryx)
I20260808 18:27:59.889447 19540 webserver.cc:460] Webserver started at http://127.7.164.67:40379/ using document root <none> and password file <none>
I20260808 18:27:59.890133 19540 fs_manager.cc:359] Metadata directory not provided
I20260808 18:27:59.890241 19540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:27:59.890463 19540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:27:59.892566 19540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "ed97518de6fe471da5fae6aa597806de"
format_stamp: "Formatted at 2026-08-08 18:27:59 on dist-test-slave-jztv"
server_key: "805975374062d3a8e3fabf033fcb0eb8"
server_key_iv: "9c9dc0fc715c810552e35ad97e7aa0ec"
server_key_version: "encryptionkey@0"
I20260808 18:27:59.893139 19540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "ed97518de6fe471da5fae6aa597806de"
format_stamp: "Formatted at 2026-08-08 18:27:59 on dist-test-slave-jztv"
server_key: "805975374062d3a8e3fabf033fcb0eb8"
server_key_iv: "9c9dc0fc715c810552e35ad97e7aa0ec"
server_key_version: "encryptionkey@0"
I20260808 18:27:59.896852 19540 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260808 18:27:59.899295 19562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:59.900537 19540 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:27:59.900630 19540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "ed97518de6fe471da5fae6aa597806de"
format_stamp: "Formatted at 2026-08-08 18:27:59 on dist-test-slave-jztv"
server_key: "805975374062d3a8e3fabf033fcb0eb8"
server_key_iv: "9c9dc0fc715c810552e35ad97e7aa0ec"
server_key_version: "encryptionkey@0"
I20260808 18:27:59.900713 19540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:27:59.916851 19540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:27:59.919683 19540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:27:59.919834 19540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:27:59.921267 19540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:27:59.921320 19540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:27:59.921362 19540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:27:59.921415 19540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:27:59 dist-test-slave-jztv krb5kdc[18983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213679, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
I20260808 18:27:59.932649 19540 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:38335
I20260808 18:27:59.933444 19737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:38335 every 8 connection(s)
I20260808 18:27:59.933514 19540 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:27:59.935585 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.921979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:50451 (local address 127.7.164.126:32975)
0808 18:27:59.922151 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:59.922156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:59.923092 (+ 936us) server_negotiation.cc:407] Connection header received
0808 18:27:59.924173 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:59.924179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:59.924245 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:59.924338 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:59.925882 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.926814 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.928093 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.928420 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.928558 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.931627 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:59.931631 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:59.931634 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:59.931662 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:59.933247 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.933848 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.933851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.933853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.933973 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.934426 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.934430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.934432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.934627 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:59.934741 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:59.935289 (+ 548us) server_negotiation.cc:299] Negotiation successful
0808 18:27:59.935416 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 18:27:59.936371 19580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.922475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:32975 (local address 127.7.164.67:50451)
0808 18:27:59.922923 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:59.922972 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:59.923917 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:59.924795 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:59.924802 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:59.925225 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:59.925717 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.925731 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.926961 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.926965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.927958 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.927968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.928630 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.928637 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.928883 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.929581 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:59.929605 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:59.931447 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:59.933369 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:59.933377 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:59.933394 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:59.933739 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:59.934094 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:59.934097 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:59.934099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:59.934336 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:59.934738 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:59.934745 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:59.935048 (+ 303us) client_negotiation.cc:769] Sending connection context
0808 18:27:59.935228 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 18:27:59.935564 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":122,"threads_started":1}
I20260808 18:27:59.937664 19738 heartbeater.cc:347] Connected to a master server at 127.7.164.126:32975
I20260808 18:27:59.937851 19738 heartbeater.cc:464] Registering TS with master...
I20260808 18:27:59.938418 19738 heartbeater.cc:511] Master 127.7.164.126:32975 requested a full tablet report, sending...
I20260808 18:27:59.939441 19046 ts_manager.cc:194] Registered new tserver with Master: ed97518de6fe471da5fae6aa597806de (127.7.164.67:38335)
I20260808 18:27:59.939860 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 19540
I20260808 18:27:59.939996 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:27:59.940238 19046 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:50451
I20260808 18:27:59.940416 7825 external_mini_cluster.cc:1428] Setting key aa735f1d6a48f982c9d0952915e12492
I20260808 18:27:59.942631 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:27:59.947734 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.940077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:45395 (local address 127.7.164.126:32975)
0808 18:27:59.940194 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:59.940199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:59.940246 (+ 47us) server_negotiation.cc:407] Connection header received
0808 18:27:59.940413 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:59.940416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:59.940460 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:59.940525 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:59.941160 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.941895 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.942997 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.943338 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.943457 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.944488 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:59.944494 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:59.944500 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:59.944531 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:59.946001 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.946580 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.946583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.946584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.946655 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.946960 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.946962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.946963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.947119 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:59.947192 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:59.947522 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 18:27:59.947622 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:27:59.947922 19747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.939773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:32975 (local address 127.7.164.67:45395)
0808 18:27:59.940165 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:59.940179 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:59.940293 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:59.940671 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:59.940673 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:59.940914 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:27:59.941042 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.941049 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.942062 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.942065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.942886 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.942896 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.943464 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.943467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.943549 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.944038 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:27:59.944049 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:27:59.944345 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:27:59.946117 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:59.946123 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:59.946126 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:59.946468 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:59.946751 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:27:59.946757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:27:59.946761 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:27:59.946866 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:27:59.947215 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:27:59.947221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:27:59.947337 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 18:27:59.947459 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 18:27:59.947641 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":117,"threads_started":1}
Aug 08 18:27:59 dist-test-slave-jztv krb5kdc[18983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213679, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:27:59 dist-test-slave-jztv krb5kdc[18983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:27:59 dist-test-slave-jztv krb5kdc[18983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 18:27:59.969925 19112 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:27:59.963440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46072 (local address 127.7.164.126:32975)
0808 18:27:59.963587 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:59.963591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:59.963603 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:27:59.963644 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:59.963647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:59.963685 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:59.963781 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:59.964556 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.965091 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.965976 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.966198 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.966370 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.969728 (+ 3358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46072: BlockingRecv error: recv got EOF from 127.0.0.1:46072 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 18:27:59 dist-test-slave-jztv krb5kdc[18983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213679, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
I20260808 18:27:59.979873 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.971659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46076 (local address 127.7.164.126:32975)
0808 18:27:59.971816 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:59.971822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:59.972249 (+ 427us) server_negotiation.cc:407] Connection header received
0808 18:27:59.972360 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:59.972362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:59.972403 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:59.972515 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:27:59.973159 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.973651 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.974533 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.974749 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.974833 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:27:59.977117 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:27:59.977121 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:27:59.977123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:27:59.977147 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:27:59.978471 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.978853 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.978856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.978857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.978931 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:27:59.979163 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:27:59.979165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:27:59.979166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:27:59.979352 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:27:59.979453 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 18:27:59.979658 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 18:27:59.979753 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 18:27:59.982820 19046 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:27:59.985819 19046 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:28:00.000835 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.995306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:33565 (local address 127.0.0.1:56756)
0808 18:27:59.996235 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:59.996256 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:59.996358 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:59.996818 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:59.996824 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:59.996848 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:59.996965 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.996972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.998431 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.998434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.999376 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.999387 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.000262 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.000266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.000388 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.000429 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 18:28:00.000554 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 18:28:00.000683 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":24,"thread_start_us":95,"threads_started":1}
I20260808 18:28:00.000835 19774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.995735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56756 (local address 127.7.164.65:33565)
0808 18:27:59.996118 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:59.996125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:59.996341 (+ 216us) server_negotiation.cc:407] Connection header received
0808 18:27:59.996470 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:59.996477 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:59.996672 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:59.996826 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:59.997097 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.998295 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.999522 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.000127 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.000262 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.000442 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.000584 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.000684 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":79,"threads_started":1}
I20260808 18:28:00.001250 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.994702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:38335 (local address 127.0.0.1:48328)
0808 18:27:59.996006 (+ 1304us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:59.996021 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:59.996113 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:59.997052 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:59.997055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:59.997075 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:59.997162 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.997166 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.998370 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.998373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.999523 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.999532 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.000670 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.000674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.000898 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.000921 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:28:00.001001 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 18:28:00.001091 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1234,"mutex_wait_us":72,"thread_start_us":112,"threads_started":1}
I20260808 18:28:00.001761 19773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.995564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48328 (local address 127.7.164.67:38335)
0808 18:27:59.996498 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:59.996517 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:59.996535 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:27:59.996594 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:59.996598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:59.996733 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:59.996895 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:59.997273 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.998203 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.999619 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.000349 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.000442 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.000598 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.001501 (+ 903us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.001622 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":125,"threads_started":1}
I20260808 18:28:00.001793 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.994764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55874 (local address 127.7.164.66:38205)
0808 18:27:59.995268 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0808 18:27:59.995289 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 18:27:59.995705 (+ 416us) server_negotiation.cc:407] Connection header received
0808 18:27:59.995790 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:27:59.995796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:27:59.996011 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:27:59.996177 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:27:59.996499 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.997701 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.999469 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.000311 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.000431 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.000598 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.001568 (+ 970us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.001660 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":144,"threads_started":1}
I20260808 18:28:00.002038 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:27:59.994583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:38205 (local address 127.0.0.1:55874)
0808 18:27:59.995349 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0808 18:27:59.995388 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:27:59.995550 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:27:59.996169 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:27:59.996175 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:27:59.996199 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:27:59.996346 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.996355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:27:59.997842 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:27:59.997845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:27:59.999317 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:27:59.999341 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.000671 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.000674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.000894 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.001098 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 18:28:00.001161 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 18:28:00.001220 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":56,"thread_start_us":74,"threads_started":1}
I20260808 18:28:00.004065 19233 tablet_service.cc:1467] Processing CreateTablet for tablet 7aa5bd3243db4663823d185274327975 (DEFAULT_TABLE table=test-table [id=d8922a8a98604bcab28bf73af711e290]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:00.004074 19639 tablet_service.cc:1467] Processing CreateTablet for tablet 7aa5bd3243db4663823d185274327975 (DEFAULT_TABLE table=test-table [id=d8922a8a98604bcab28bf73af711e290]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:00.005163 19233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7aa5bd3243db4663823d185274327975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:00.005167 19639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7aa5bd3243db4663823d185274327975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:00.006807 19430 tablet_service.cc:1467] Processing CreateTablet for tablet 7aa5bd3243db4663823d185274327975 (DEFAULT_TABLE table=test-table [id=d8922a8a98604bcab28bf73af711e290]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:00.007855 19430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7aa5bd3243db4663823d185274327975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:00.011714 19775 tablet_bootstrap.cc:492] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257: Bootstrap starting.
I20260808 18:28:00.013166 19777 tablet_bootstrap.cc:492] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de: Bootstrap starting.
I20260808 18:28:00.014245 19775 tablet_bootstrap.cc:654] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:00.015269 19775 log.cc:824] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257: Log is configured to *not* fsync() on all Append() calls
I20260808 18:28:00.015699 19777 tablet_bootstrap.cc:654] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:00.016008 19778 tablet_bootstrap.cc:492] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5: Bootstrap starting.
I20260808 18:28:00.016638 19777 log.cc:824] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de: Log is configured to *not* fsync() on all Append() calls
I20260808 18:28:00.017570 19775 tablet_bootstrap.cc:492] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257: No bootstrap required, opened a new log
I20260808 18:28:00.017768 19775 ts_tablet_manager.cc:1397] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:28:00.018643 19778 tablet_bootstrap.cc:654] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:00.018669 19777 tablet_bootstrap.cc:492] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de: No bootstrap required, opened a new log
I20260808 18:28:00.018848 19777 ts_tablet_manager.cc:1397] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:28:00.019454 19778 log.cc:824] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5: Log is configured to *not* fsync() on all Append() calls
I20260808 18:28:00.021142 19778 tablet_bootstrap.cc:492] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5: No bootstrap required, opened a new log
I20260808 18:28:00.021363 19778 ts_tablet_manager.cc:1397] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260808 18:28:00.022517 19775 raft_consensus.cc:348] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } }
I20260808 18:28:00.022799 19775 raft_consensus.cc:374] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:00.022861 19775 raft_consensus.cc:729] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892d0db548c4449c8d67d8f629d38257, State: Initialized, Role: FOLLOWER
I20260808 18:28:00.023058 19777 raft_consensus.cc:348] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } }
I20260808 18:28:00.023308 19777 raft_consensus.cc:374] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:00.023368 19777 raft_consensus.cc:729] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed97518de6fe471da5fae6aa597806de, State: Initialized, Role: FOLLOWER
I20260808 18:28:00.023319 19775 consensus_queue.cc:260] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } }
I20260808 18:28:00.023833 19777 consensus_queue.cc:260] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } }
I20260808 18:28:00.024333 19775 ts_tablet_manager.cc:1428] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 18:28:00.024595 19738 heartbeater.cc:503] Master 127.7.164.126:32975 was elected leader, sending a full tablet report...
I20260808 18:28:00.024673 19329 heartbeater.cc:503] Master 127.7.164.126:32975 was elected leader, sending a full tablet report...
I20260808 18:28:00.024637 19778 raft_consensus.cc:348] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } }
I20260808 18:28:00.024866 19778 raft_consensus.cc:374] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:00.024916 19778 raft_consensus.cc:729] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906fd6cd59e44e3db022089c7c1b14b5, State: Initialized, Role: FOLLOWER
I20260808 18:28:00.025498 19778 consensus_queue.cc:260] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } }
I20260808 18:28:00.026171 19777 ts_tablet_manager.cc:1428] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 18:28:00.026392 19778 ts_tablet_manager.cc:1428] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260808 18:28:00.026751 19528 heartbeater.cc:503] Master 127.7.164.126:32975 was elected leader, sending a full tablet report...
W20260808 18:28:00.184633 19739 tablet.cc:2369] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:28:00.202873 19330 tablet.cc:2369] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:28:00.206641 19529 tablet.cc:2369] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:28:00.265043 19783 raft_consensus.cc:482] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:28:00.265291 19783 raft_consensus.cc:504] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } }
I20260808 18:28:00.266896 19783 leader_election.cc:288] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 906fd6cd59e44e3db022089c7c1b14b5 (127.7.164.66:38205), ed97518de6fe471da5fae6aa597806de (127.7.164.67:38335)
I20260808 18:28:00.269377 19788 raft_consensus.cc:482] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:28:00.269596 19788 raft_consensus.cc:504] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } }
I20260808 18:28:00.270949 19788 leader_election.cc:288] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed97518de6fe471da5fae6aa597806de (127.7.164.67:38335), 892d0db548c4449c8d67d8f629d38257 (127.7.164.65:33565)
I20260808 18:28:00.274024 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.269312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:40601 (local address 127.7.164.66:38205)
0808 18:28:00.269443 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:00.269448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:00.269517 (+ 69us) server_negotiation.cc:407] Connection header received
0808 18:28:00.269725 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:00.269729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:00.269775 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:00.269891 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:28:00.270161 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.270823 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.272734 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.273326 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.273421 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.273474 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.273820 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.273891 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:28:00.274287 19773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.268164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:36561 (local address 127.7.164.67:38335)
0808 18:28:00.268333 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:00.268340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:00.269472 (+ 1132us) server_negotiation.cc:407] Connection header received
0808 18:28:00.269755 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:00.269758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:00.269809 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:00.269858 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:28:00.270372 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.271083 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.272713 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.273451 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.273560 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.273623 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.274049 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.274146 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:28:00.274478 19789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.268064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:38205 (local address 127.7.164.65:40601)
0808 18:28:00.269429 (+ 1365us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:00.269445 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:00.269605 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:00.269906 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:00.269909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:00.269926 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:28:00.270052 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.270057 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.271382 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.271388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.272602 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.272612 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.273576 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.273580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.273680 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.273704 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:28:00.274201 (+ 497us) client_negotiation.cc:240] Negotiation successful
0808 18:28:00.274306 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1241,"spinlock_wait_cycles":14848,"thread_start_us":1043,"threads_started":1}
I20260808 18:28:00.274732 19790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.267291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:38335 (local address 127.7.164.65:36561)
0808 18:28:00.269359 (+ 2068us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:00.269377 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:00.269608 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:00.270019 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:00.270022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:00.270040 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:28:00.270160 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.270166 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.271247 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.271250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.272561 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.272575 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.273735 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.273741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.273869 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.273903 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 18:28:00.274485 (+ 582us) client_negotiation.cc:240] Negotiation successful
0808 18:28:00.274579 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1943,"mutex_wait_us":2,"spinlock_wait_cycles":42112,"thread_start_us":481,"threads_started":1}
I20260808 18:28:00.275094 19468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aa5bd3243db4663823d185274327975" candidate_uuid: "892d0db548c4449c8d67d8f629d38257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906fd6cd59e44e3db022089c7c1b14b5" is_pre_election: true
I20260808 18:28:00.275342 19468 raft_consensus.cc:2454] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 892d0db548c4449c8d67d8f629d38257 in term 0.
I20260808 18:28:00.275590 19668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aa5bd3243db4663823d185274327975" candidate_uuid: "892d0db548c4449c8d67d8f629d38257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed97518de6fe471da5fae6aa597806de" is_pre_election: true
I20260808 18:28:00.275841 19166 leader_election.cc:302] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 892d0db548c4449c8d67d8f629d38257, 906fd6cd59e44e3db022089c7c1b14b5; no voters:
I20260808 18:28:00.275900 19668 raft_consensus.cc:2454] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 892d0db548c4449c8d67d8f629d38257 in term 0.
I20260808 18:28:00.276233 19783 raft_consensus.cc:2781] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:28:00.276357 19783 raft_consensus.cc:482] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:28:00.276420 19783 raft_consensus.cc:3037] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:00.276611 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.271153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:38335 (local address 127.7.164.66:45937)
0808 18:28:00.271590 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:00.271606 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:00.271690 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:00.272092 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:00.272096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:00.272115 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:28:00.272211 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.272218 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.273218 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.273222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.274416 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.274428 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.276204 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.276207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.276318 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.276345 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:28:00.276410 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 18:28:00.276476 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":59,"threads_started":1}
I20260808 18:28:00.277014 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.271225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:45937 (local address 127.7.164.67:38335)
0808 18:28:00.271872 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:00.271879 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:00.271893 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:28:00.271932 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:00.271934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:00.271972 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:00.272064 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:28:00.272341 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.273077 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.274567 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.275979 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.276093 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.276141 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.276812 (+ 671us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.276892 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":84,"threads_started":1}
I20260808 18:28:00.278012 19783 raft_consensus.cc:504] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } }
I20260808 18:28:00.278901 19668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aa5bd3243db4663823d185274327975" candidate_uuid: "906fd6cd59e44e3db022089c7c1b14b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed97518de6fe471da5fae6aa597806de" is_pre_election: true
I20260808 18:28:00.279052 19668 raft_consensus.cc:2454] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 906fd6cd59e44e3db022089c7c1b14b5 in term 0.
I20260808 18:28:00.279549 19783 leader_election.cc:288] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [CANDIDATE]: Term 1 election: Requested vote from peers 906fd6cd59e44e3db022089c7c1b14b5 (127.7.164.66:38205), ed97518de6fe471da5fae6aa597806de (127.7.164.67:38335)
I20260808 18:28:00.279506 19366 leader_election.cc:302] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 906fd6cd59e44e3db022089c7c1b14b5, ed97518de6fe471da5fae6aa597806de; no voters:
I20260808 18:28:00.279506 19793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.271286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:33565 (local address 127.7.164.66:40901)
0808 18:28:00.275043 (+ 3757us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:00.275056 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:00.275183 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:00.275531 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:00.275533 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:00.275549 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:28:00.276220 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.276227 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.277176 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.277179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.278221 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.278258 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.279101 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.279103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.279175 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.279189 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 18:28:00.279275 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 18:28:00.279380 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3698,"thread_start_us":82,"threads_started":1}
I20260808 18:28:00.279810 19788 raft_consensus.cc:2781] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:28:00.280025 19774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.271624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:40901 (local address 127.7.164.65:33565)
0808 18:28:00.271727 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:00.271732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:00.275123 (+ 3391us) server_negotiation.cc:407] Connection header received
0808 18:28:00.275298 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:00.275301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:00.275343 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:00.275405 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:28:00.276342 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.277030 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.278407 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.278862 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.278977 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.279022 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.279835 (+ 813us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.279918 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 18:28:00.280087 19788 raft_consensus.cc:482] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:28:00.280105 19668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aa5bd3243db4663823d185274327975" candidate_uuid: "892d0db548c4449c8d67d8f629d38257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed97518de6fe471da5fae6aa597806de"
I20260808 18:28:00.280159 19788 raft_consensus.cc:3037] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:00.280207 19668 raft_consensus.cc:3037] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:00.281400 19668 raft_consensus.cc:2454] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 892d0db548c4449c8d67d8f629d38257 in term 1.
I20260808 18:28:00.281409 19788 raft_consensus.cc:504] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } }
I20260808 18:28:00.281759 19788 leader_election.cc:288] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [CANDIDATE]: Term 1 election: Requested vote from peers ed97518de6fe471da5fae6aa597806de (127.7.164.67:38335), 892d0db548c4449c8d67d8f629d38257 (127.7.164.65:33565)
I20260808 18:28:00.281756 19167 leader_election.cc:302] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 892d0db548c4449c8d67d8f629d38257, ed97518de6fe471da5fae6aa597806de; no voters:
I20260808 18:28:00.281939 19783 raft_consensus.cc:2781] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:28:00.282120 19783 raft_consensus.cc:686] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [term 1 LEADER]: Becoming Leader. State: Replica: 892d0db548c4449c8d67d8f629d38257, State: Running, Role: LEADER
I20260808 18:28:00.282145 19668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aa5bd3243db4663823d185274327975" candidate_uuid: "906fd6cd59e44e3db022089c7c1b14b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed97518de6fe471da5fae6aa597806de"
I20260808 18:28:00.282334 19668 raft_consensus.cc:2379] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 906fd6cd59e44e3db022089c7c1b14b5 in current term 1: Already voted for candidate 892d0db548c4449c8d67d8f629d38257 in this term.
I20260808 18:28:00.279954 19468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aa5bd3243db4663823d185274327975" candidate_uuid: "892d0db548c4449c8d67d8f629d38257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906fd6cd59e44e3db022089c7c1b14b5"
I20260808 18:28:00.282497 19468 raft_consensus.cc:2379] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 892d0db548c4449c8d67d8f629d38257 in current term 1: Already voted for candidate 906fd6cd59e44e3db022089c7c1b14b5 in this term.
I20260808 18:28:00.282502 19783 consensus_queue.cc:237] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } }
I20260808 18:28:00.282578 19263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aa5bd3243db4663823d185274327975" candidate_uuid: "906fd6cd59e44e3db022089c7c1b14b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892d0db548c4449c8d67d8f629d38257"
I20260808 18:28:00.283113 19265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aa5bd3243db4663823d185274327975" candidate_uuid: "906fd6cd59e44e3db022089c7c1b14b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892d0db548c4449c8d67d8f629d38257" is_pre_election: true
I20260808 18:28:00.283183 19368 leader_election.cc:302] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 906fd6cd59e44e3db022089c7c1b14b5; no voters: 892d0db548c4449c8d67d8f629d38257, ed97518de6fe471da5fae6aa597806de
I20260808 18:28:00.283366 19788 raft_consensus.cc:2726] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 18:28:00.286530 19045 catalog_manager.cc:5495] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 reported cstate change: term changed from 0 to 1, leader changed from <none> to 892d0db548c4449c8d67d8f629d38257 (127.7.164.65). New cstate: current_term: 1 leader_uuid: "892d0db548c4449c8d67d8f629d38257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } health_report { overall_health: HEALTHY } } }
I20260808 18:28:00.306180 19774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.301244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56770 (local address 127.7.164.65:33565)
0808 18:28:00.301419 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:00.301425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:00.301445 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:28:00.301581 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:00.301586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:00.301673 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:00.301828 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:28:00.302026 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.302933 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.304187 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.304592 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.304721 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.305019 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:28:00.305686 (+ 667us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:28:00.305824 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.305952 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.306026 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:28:00.312340 19468 raft_consensus.cc:1261] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Refusing update from remote peer 892d0db548c4449c8d67d8f629d38257: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:28:00.312387 19668 raft_consensus.cc:1261] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Refusing update from remote peer 892d0db548c4449c8d67d8f629d38257: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:28:00.312963 19783 consensus_queue.cc:1035] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:28:00.313490 19794 consensus_queue.cc:1035] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:28:00.324018 19802 mvcc.cc:205] Tried to move back new op lower bound from 7316331234552459264 to 7316331234441687040. Current Snapshot: MvccSnapshot[applied={T|T < 7316331234552459264}]
I20260808 18:28:00.330499 19044 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:28:00.330658 19044 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:28:00.333117 19044 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9: Sending DeleteTablet for 3 replicas of tablet 7aa5bd3243db4663823d185274327975
I20260808 18:28:00.333923 19639 tablet_service.cc:1514] Processing DeleteTablet for tablet 7aa5bd3243db4663823d185274327975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:00 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:48328
I20260808 18:28:00.333904 19233 tablet_service.cc:1514] Processing DeleteTablet for tablet 7aa5bd3243db4663823d185274327975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:00 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:56756
I20260808 18:28:00.333937 19430 tablet_service.cc:1514] Processing DeleteTablet for tablet 7aa5bd3243db4663823d185274327975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:00 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:55874
I20260808 18:28:00.334553 19812 tablet_replica.cc:330] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de: stopping tablet replica
I20260808 18:28:00.334970 19812 raft_consensus.cc:2229] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:28:00.335318 19812 raft_consensus.cc:2258] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:28:00.335484 19044 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:28:00.335945 19044 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:28:00.337193 19812 ts_tablet_manager.cc:1905] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:28:00.340085 19812 ts_tablet_manager.cc:1918] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:28:00.340188 19812 log.cc:1196] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/7aa5bd3243db4663823d185274327975
I20260808 18:28:00.340517 19812 ts_tablet_manager.cc:1939] T 7aa5bd3243db4663823d185274327975 P ed97518de6fe471da5fae6aa597806de: Deleting consensus metadata
I20260808 18:28:00.340873 19639 tablet_service.cc:1467] Processing CreateTablet for tablet 8b18f057d5714c2298e6515c0ee910b5 (DEFAULT_TABLE table=test-table [id=da9dd840cd0c40d78a203ddd59129f11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:00.341216 19639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b18f057d5714c2298e6515c0ee910b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:00.341473 19430 tablet_service.cc:1467] Processing CreateTablet for tablet 8b18f057d5714c2298e6515c0ee910b5 (DEFAULT_TABLE table=test-table [id=da9dd840cd0c40d78a203ddd59129f11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:00.341744 19430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b18f057d5714c2298e6515c0ee910b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:00.341718 19028 catalog_manager.cc:4841] TS ed97518de6fe471da5fae6aa597806de (127.7.164.67:38335): tablet 7aa5bd3243db4663823d185274327975 (table test-table [id=d8922a8a98604bcab28bf73af711e290]) successfully deleted
I20260808 18:28:00.342028 19815 tablet_replica.cc:330] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5: stopping tablet replica
I20260808 18:28:00.342356 19815 raft_consensus.cc:2229] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:28:00.342653 19815 raft_consensus.cc:2258] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:28:00.335726 19813 tablet_replica.cc:330] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257: stopping tablet replica
I20260808 18:28:00.343142 19813 raft_consensus.cc:2229] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:28:00.343380 19777 tablet_bootstrap.cc:492] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de: Bootstrap starting.
I20260808 18:28:00.343510 19813 raft_consensus.cc:2258] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:28:00.344358 19777 tablet_bootstrap.cc:654] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:00.344976 19813 ts_tablet_manager.cc:1905] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:28:00.345041 19815 ts_tablet_manager.cc:1905] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:28:00.345239 19233 tablet_service.cc:1467] Processing CreateTablet for tablet 8b18f057d5714c2298e6515c0ee910b5 (DEFAULT_TABLE table=test-table [id=da9dd840cd0c40d78a203ddd59129f11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:00.345506 19233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b18f057d5714c2298e6515c0ee910b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:00.347527 19815 ts_tablet_manager.cc:1918] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:28:00.347625 19815 log.cc:1196] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/7aa5bd3243db4663823d185274327975
I20260808 18:28:00.347721 19775 tablet_bootstrap.cc:492] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257: Bootstrap starting.
I20260808 18:28:00.347942 19815 ts_tablet_manager.cc:1939] T 7aa5bd3243db4663823d185274327975 P 906fd6cd59e44e3db022089c7c1b14b5: Deleting consensus metadata
I20260808 18:28:00.348664 19775 tablet_bootstrap.cc:654] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:00.349107 19027 catalog_manager.cc:4841] TS 906fd6cd59e44e3db022089c7c1b14b5 (127.7.164.66:38205): tablet 7aa5bd3243db4663823d185274327975 (table test-table [id=d8922a8a98604bcab28bf73af711e290]) successfully deleted
I20260808 18:28:00.349359 19813 ts_tablet_manager.cc:1918] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:28:00.349560 19813 log.cc:1196] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/wal/wals/7aa5bd3243db4663823d185274327975
I20260808 18:28:00.349956 19813 ts_tablet_manager.cc:1939] T 7aa5bd3243db4663823d185274327975 P 892d0db548c4449c8d67d8f629d38257: Deleting consensus metadata
I20260808 18:28:00.350056 19775 tablet_bootstrap.cc:492] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257: No bootstrap required, opened a new log
I20260808 18:28:00.350158 19775 ts_tablet_manager.cc:1397] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:28:00.350631 19775 raft_consensus.cc:348] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.350768 19775 raft_consensus.cc:374] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:00.350805 19775 raft_consensus.cc:729] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892d0db548c4449c8d67d8f629d38257, State: Initialized, Role: FOLLOWER
I20260808 18:28:00.350919 19775 consensus_queue.cc:260] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.351184 19775 ts_tablet_manager.cc:1428] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 18:28:00.351390 19778 tablet_bootstrap.cc:492] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5: Bootstrap starting.
I20260808 18:28:00.351800 19777 tablet_bootstrap.cc:492] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de: No bootstrap required, opened a new log
I20260808 18:28:00.351922 19777 ts_tablet_manager.cc:1397] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260808 18:28:00.352309 19778 tablet_bootstrap.cc:654] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:00.352305 19777 raft_consensus.cc:348] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.352438 19777 raft_consensus.cc:374] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:00.352478 19777 raft_consensus.cc:729] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed97518de6fe471da5fae6aa597806de, State: Initialized, Role: FOLLOWER
I20260808 18:28:00.352594 19777 consensus_queue.cc:260] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.352838 19777 ts_tablet_manager.cc:1428] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:28:00.353246 19031 catalog_manager.cc:4841] TS 892d0db548c4449c8d67d8f629d38257 (127.7.164.65:33565): tablet 7aa5bd3243db4663823d185274327975 (table test-table [id=d8922a8a98604bcab28bf73af711e290]) successfully deleted
I20260808 18:28:00.354019 19778 tablet_bootstrap.cc:492] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5: No bootstrap required, opened a new log
I20260808 18:28:00.354103 19778 ts_tablet_manager.cc:1397] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 18:28:00.354532 19778 raft_consensus.cc:348] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.354645 19778 raft_consensus.cc:374] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:00.354674 19778 raft_consensus.cc:729] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906fd6cd59e44e3db022089c7c1b14b5, State: Initialized, Role: FOLLOWER
I20260808 18:28:00.354789 19778 consensus_queue.cc:260] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.355054 19778 ts_tablet_manager.cc:1428] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 18:28:00.391703 19787 raft_consensus.cc:482] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:28:00.391849 19787 raft_consensus.cc:504] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.393059 19787 leader_election.cc:288] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 892d0db548c4449c8d67d8f629d38257 (127.7.164.65:33565), 906fd6cd59e44e3db022089c7c1b14b5 (127.7.164.66:38205)
I20260808 18:28:00.397552 19580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.393273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:33565 (local address 127.7.164.67:37463)
0808 18:28:00.393548 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:00.393565 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:00.393667 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:00.394020 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:00.394023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:00.394038 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:28:00.394162 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.394168 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.395012 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.395018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.396192 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.396203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.397085 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.397089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.397206 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.397232 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 18:28:00.397309 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 18:28:00.397390 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199}
I20260808 18:28:00.397555 19774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.393360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:37463 (local address 127.7.164.65:33565)
0808 18:28:00.393511 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:00.393516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:00.393759 (+ 243us) server_negotiation.cc:407] Connection header received
0808 18:28:00.393827 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:00.393830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:00.393891 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:00.393990 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:28:00.394296 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.394898 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.396339 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.396953 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.397089 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.397151 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.397339 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.397419 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:28:00.397845 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.393634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:60147 (local address 127.7.164.66:38205)
0808 18:28:00.393770 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:00.393776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:00.393825 (+ 49us) server_negotiation.cc:407] Connection header received
0808 18:28:00.393991 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:00.393994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:00.394041 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:00.394120 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:28:00.394532 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.395291 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.396659 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.397124 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.397238 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.397304 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.397608 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.397701 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:28:00.398324 19819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.393331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:38205 (local address 127.7.164.67:60147)
0808 18:28:00.393740 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:00.393754 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:00.393863 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:00.394257 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:00.394260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:00.394280 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:28:00.394400 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.394406 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.395413 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.395417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.396537 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.396545 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.397360 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.397363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.397447 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.397468 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:28:00.398066 (+ 598us) client_negotiation.cc:240] Negotiation successful
0808 18:28:00.398141 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":88,"threads_started":1}
I20260808 18:28:00.398437 19265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b18f057d5714c2298e6515c0ee910b5" candidate_uuid: "ed97518de6fe471da5fae6aa597806de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892d0db548c4449c8d67d8f629d38257" is_pre_election: true
I20260808 18:28:00.398660 19265 raft_consensus.cc:2454] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed97518de6fe471da5fae6aa597806de in term 0.
I20260808 18:28:00.398833 19468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b18f057d5714c2298e6515c0ee910b5" candidate_uuid: "ed97518de6fe471da5fae6aa597806de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906fd6cd59e44e3db022089c7c1b14b5" is_pre_election: true
I20260808 18:28:00.398970 19468 raft_consensus.cc:2454] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed97518de6fe471da5fae6aa597806de in term 0.
I20260808 18:28:00.399116 19571 leader_election.cc:302] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 892d0db548c4449c8d67d8f629d38257, ed97518de6fe471da5fae6aa597806de; no voters:
I20260808 18:28:00.399428 19787 raft_consensus.cc:2781] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:28:00.399489 19787 raft_consensus.cc:482] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:28:00.399534 19787 raft_consensus.cc:3037] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:00.400799 19787 raft_consensus.cc:504] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.401226 19787 leader_election.cc:288] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [CANDIDATE]: Term 1 election: Requested vote from peers 892d0db548c4449c8d67d8f629d38257 (127.7.164.65:33565), 906fd6cd59e44e3db022089c7c1b14b5 (127.7.164.66:38205)
I20260808 18:28:00.401608 19265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b18f057d5714c2298e6515c0ee910b5" candidate_uuid: "ed97518de6fe471da5fae6aa597806de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892d0db548c4449c8d67d8f629d38257"
I20260808 18:28:00.401620 19468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b18f057d5714c2298e6515c0ee910b5" candidate_uuid: "ed97518de6fe471da5fae6aa597806de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906fd6cd59e44e3db022089c7c1b14b5"
I20260808 18:28:00.401737 19468 raft_consensus.cc:3037] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:00.401746 19265 raft_consensus.cc:3037] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:00.402810 19265 raft_consensus.cc:2454] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed97518de6fe471da5fae6aa597806de in term 1.
I20260808 18:28:00.402921 19468 raft_consensus.cc:2454] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed97518de6fe471da5fae6aa597806de in term 1.
I20260808 18:28:00.403152 19571 leader_election.cc:302] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 892d0db548c4449c8d67d8f629d38257, ed97518de6fe471da5fae6aa597806de; no voters:
I20260808 18:28:00.403342 19787 raft_consensus.cc:2781] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:28:00.403584 19787 raft_consensus.cc:686] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [term 1 LEADER]: Becoming Leader. State: Replica: ed97518de6fe471da5fae6aa597806de, State: Running, Role: LEADER
I20260808 18:28:00.403880 19787 consensus_queue.cc:237] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.406250 19045 catalog_manager.cc:5495] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de reported cstate change: term changed from 0 to 1, leader changed from <none> to ed97518de6fe471da5fae6aa597806de (127.7.164.67). New cstate: current_term: 1 leader_uuid: "ed97518de6fe471da5fae6aa597806de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } health_report { overall_health: HEALTHY } } }
Aug 08 18:28:00 dist-test-slave-jztv krb5kdc[18983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213679, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.126@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
I20260808 18:28:00.446309 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.436422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46092 (local address 127.7.164.126:32975)
0808 18:28:00.436578 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:00.436583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:00.436797 (+ 214us) server_negotiation.cc:407] Connection header received
0808 18:28:00.436976 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:00.436979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:00.437038 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:00.437109 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:00.438083 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.438677 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.439879 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.440110 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.440193 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.442911 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:00.442915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:00.442917 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:28:00.442939 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:00.444468 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:00.444967 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:00.444972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:00.444974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:00.445057 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:00.445333 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:00.445338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:00.445340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:00.445538 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:00.445644 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.445938 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.446149 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":61}
I20260808 18:28:00.446326 19829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.436352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:32975 (local address 127.0.0.1:46092)
0808 18:28:00.436694 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:00.436710 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:00.436846 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:00.437185 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:00.437189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:00.437771 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:00.437903 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.437911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.438842 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.438849 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.439745 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.439757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.440239 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.440243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.440336 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.440981 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:00.440997 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:00.442752 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:28:00.444600 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:28:00.444612 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:28:00.444618 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:28:00.444862 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:28:00.445150 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:28:00.445155 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:28:00.445157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:28:00.445243 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:28:00.445644 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:28:00.445652 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:28:00.445771 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 18:28:00.445891 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 18:28:00.446084 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":129,"threads_started":1}
I20260808 18:28:00.450724 19045 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:46092:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 18:28:00.451560 19045 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:28:00.457113 19233 tablet_service.cc:1467] Processing CreateTablet for tablet 46a51d6c9688468b8a31f54b7f86ced4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c9f106eff037415484d522626d4b2808]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:28:00.457530 19233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46a51d6c9688468b8a31f54b7f86ced4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:00.460063 19775 tablet_bootstrap.cc:492] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257: Bootstrap starting.
I20260808 18:28:00.460417 19430 tablet_service.cc:1467] Processing CreateTablet for tablet 46a51d6c9688468b8a31f54b7f86ced4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c9f106eff037415484d522626d4b2808]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:28:00.460731 19430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46a51d6c9688468b8a31f54b7f86ced4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:00.461550 19775 tablet_bootstrap.cc:654] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:00.463069 19775 tablet_bootstrap.cc:492] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257: No bootstrap required, opened a new log
I20260808 18:28:00.463177 19775 ts_tablet_manager.cc:1397] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260808 18:28:00.463673 19775 raft_consensus.cc:348] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.464048 19778 tablet_bootstrap.cc:492] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5: Bootstrap starting.
I20260808 18:28:00.465140 19778 tablet_bootstrap.cc:654] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:00.463835 19775 raft_consensus.cc:374] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:00.466024 19775 raft_consensus.cc:729] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892d0db548c4449c8d67d8f629d38257, State: Initialized, Role: FOLLOWER
I20260808 18:28:00.466199 19775 consensus_queue.cc:260] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.466508 19783 tablet_replica.cc:439] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } }
I20260808 18:28:00.466467 19639 tablet_service.cc:1467] Processing CreateTablet for tablet 46a51d6c9688468b8a31f54b7f86ced4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c9f106eff037415484d522626d4b2808]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:28:00.466639 19783 tablet_replica.cc:442] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:28:00.466871 19639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46a51d6c9688468b8a31f54b7f86ced4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:00.466998 19778 tablet_bootstrap.cc:492] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5: No bootstrap required, opened a new log
I20260808 18:28:00.467100 19778 ts_tablet_manager.cc:1397] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 18:28:00.467557 19778 raft_consensus.cc:348] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.467667 19778 raft_consensus.cc:374] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:00.467702 19778 raft_consensus.cc:729] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906fd6cd59e44e3db022089c7c1b14b5, State: Initialized, Role: FOLLOWER
I20260808 18:28:00.467854 19778 consensus_queue.cc:260] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.468132 19788 tablet_replica.cc:439] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } }
I20260808 18:28:00.468264 19788 tablet_replica.cc:442] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:28:00.468521 19778 ts_tablet_manager.cc:1428] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260808 18:28:00.468688 19775 ts_tablet_manager.cc:1428] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20260808 18:28:00.472358 19777 tablet_bootstrap.cc:492] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de: Bootstrap starting.
I20260808 18:28:00.473421 19777 tablet_bootstrap.cc:654] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:00.475159 19777 tablet_bootstrap.cc:492] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de: No bootstrap required, opened a new log
I20260808 18:28:00.475256 19777 ts_tablet_manager.cc:1397] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260808 18:28:00.475636 19777 raft_consensus.cc:348] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.475720 19777 raft_consensus.cc:374] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:00.475740 19777 raft_consensus.cc:729] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed97518de6fe471da5fae6aa597806de, State: Initialized, Role: FOLLOWER
I20260808 18:28:00.475881 19777 consensus_queue.cc:260] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.476117 19787 tablet_replica.cc:439] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } }
I20260808 18:28:00.476258 19787 tablet_replica.cc:442] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:28:00.476186 19777 ts_tablet_manager.cc:1428] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:28:00.671665 19787 raft_consensus.cc:482] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:28:00.671844 19787 raft_consensus.cc:504] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.672292 19787 leader_election.cc:288] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 892d0db548c4449c8d67d8f629d38257 (127.7.164.65:33565), 906fd6cd59e44e3db022089c7c1b14b5 (127.7.164.66:38205)
I20260808 18:28:00.672796 19265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46a51d6c9688468b8a31f54b7f86ced4" candidate_uuid: "ed97518de6fe471da5fae6aa597806de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892d0db548c4449c8d67d8f629d38257" is_pre_election: true
I20260808 18:28:00.672798 19468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46a51d6c9688468b8a31f54b7f86ced4" candidate_uuid: "ed97518de6fe471da5fae6aa597806de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906fd6cd59e44e3db022089c7c1b14b5" is_pre_election: true
I20260808 18:28:00.672945 19468 raft_consensus.cc:2454] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed97518de6fe471da5fae6aa597806de in term 0.
I20260808 18:28:00.672986 19265 raft_consensus.cc:2454] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed97518de6fe471da5fae6aa597806de in term 0.
I20260808 18:28:00.673313 19565 leader_election.cc:302] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 906fd6cd59e44e3db022089c7c1b14b5, ed97518de6fe471da5fae6aa597806de; no voters:
I20260808 18:28:00.673542 19787 raft_consensus.cc:2781] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:28:00.673604 19787 raft_consensus.cc:482] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:28:00.673630 19787 raft_consensus.cc:3037] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:00.674849 19787 raft_consensus.cc:504] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.675181 19787 leader_election.cc:288] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [CANDIDATE]: Term 1 election: Requested vote from peers 892d0db548c4449c8d67d8f629d38257 (127.7.164.65:33565), 906fd6cd59e44e3db022089c7c1b14b5 (127.7.164.66:38205)
I20260808 18:28:00.675580 19468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46a51d6c9688468b8a31f54b7f86ced4" candidate_uuid: "ed97518de6fe471da5fae6aa597806de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906fd6cd59e44e3db022089c7c1b14b5"
I20260808 18:28:00.675581 19265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46a51d6c9688468b8a31f54b7f86ced4" candidate_uuid: "ed97518de6fe471da5fae6aa597806de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892d0db548c4449c8d67d8f629d38257"
I20260808 18:28:00.675724 19468 raft_consensus.cc:3037] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:00.675735 19265 raft_consensus.cc:3037] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:00.676684 19468 raft_consensus.cc:2454] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed97518de6fe471da5fae6aa597806de in term 1.
I20260808 18:28:00.676689 19265 raft_consensus.cc:2454] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed97518de6fe471da5fae6aa597806de in term 1.
I20260808 18:28:00.677019 19565 leader_election.cc:302] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 906fd6cd59e44e3db022089c7c1b14b5, ed97518de6fe471da5fae6aa597806de; no voters:
I20260808 18:28:00.677237 19787 raft_consensus.cc:2781] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:28:00.677358 19787 raft_consensus.cc:686] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [term 1 LEADER]: Becoming Leader. State: Replica: ed97518de6fe471da5fae6aa597806de, State: Running, Role: LEADER
I20260808 18:28:00.677495 19787 consensus_queue.cc:237] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.678186 19821 tablet_replica.cc:439] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de: TxnStatusTablet state changed. Reason: New leader ed97518de6fe471da5fae6aa597806de. Latest consensus state: current_term: 1 leader_uuid: "ed97518de6fe471da5fae6aa597806de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } }
I20260808 18:28:00.678375 19821 tablet_replica.cc:442] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:28:00.678820 19835 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 18:28:00.679015 19835 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 18:28:00.679471 19046 catalog_manager.cc:5495] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de reported cstate change: term changed from 0 to 1, leader changed from <none> to ed97518de6fe471da5fae6aa597806de (127.7.164.67). New cstate: current_term: 1 leader_uuid: "ed97518de6fe471da5fae6aa597806de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } health_report { overall_health: HEALTHY } } }
I20260808 18:28:00.717626 19829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.712987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:38335 (local address 127.0.0.1:48332)
0808 18:28:00.713257 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:00.713270 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:00.713406 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:00.713784 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:00.713787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:00.713795 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:28:00.713907 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.713913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.714823 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.714829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.715716 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.715727 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.716304 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.716308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.716401 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.716457 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:28:00.717302 (+ 845us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:28:00.717320 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 18:28:00.717392 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 18:28:00.717487 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260808 18:28:00.717628 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.713075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48332 (local address 127.7.164.67:38335)
0808 18:28:00.713219 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:00.713224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:00.713346 (+ 122us) server_negotiation.cc:407] Connection header received
0808 18:28:00.713581 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:00.713584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:00.713663 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:00.713772 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:28:00.714031 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.714708 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.715839 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.716193 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.716291 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.716614 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:28:00.717205 (+ 591us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:28:00.717329 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.717413 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.717472 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:28:00.720481 19468 raft_consensus.cc:1261] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Refusing update from remote peer ed97518de6fe471da5fae6aa597806de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:28:00.720764 19265 raft_consensus.cc:1261] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Refusing update from remote peer ed97518de6fe471da5fae6aa597806de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:28:00.720986 19787 consensus_queue.cc:1035] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [LEADER]: Connected to new peer: Peer: permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:28:00.721275 19821 consensus_queue.cc:1035] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de [LEADER]: Connected to new peer: Peer: permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:28:00.722888 19788 tablet_replica.cc:439] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5: TxnStatusTablet state changed. Reason: New leader ed97518de6fe471da5fae6aa597806de. Latest consensus state: current_term: 1 leader_uuid: "ed97518de6fe471da5fae6aa597806de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } }
I20260808 18:28:00.722910 19783 tablet_replica.cc:439] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257: TxnStatusTablet state changed. Reason: New leader ed97518de6fe471da5fae6aa597806de. Latest consensus state: current_term: 1 leader_uuid: "ed97518de6fe471da5fae6aa597806de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } }
I20260808 18:28:00.723016 19788 tablet_replica.cc:442] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:28:00.723026 19783 tablet_replica.cc:442] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:28:00.725941 19787 tablet_replica.cc:439] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ed97518de6fe471da5fae6aa597806de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } }
I20260808 18:28:00.726053 19787 tablet_replica.cc:442] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:28:00.726605 19787 tablet_replica.cc:439] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ed97518de6fe471da5fae6aa597806de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } }
I20260808 18:28:00.726702 19787 tablet_replica.cc:442] T 46a51d6c9688468b8a31f54b7f86ced4 P ed97518de6fe471da5fae6aa597806de: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:28:00.726787 19799 mvcc.cc:205] Tried to move back new op lower bound from 7316331236226027520 to 7316331236056289280. Current Snapshot: MvccSnapshot[applied={T|T < 7316331236226027520}]
I20260808 18:28:00.727365 19801 mvcc.cc:205] Tried to move back new op lower bound from 7316331236226027520 to 7316331236056289280. Current Snapshot: MvccSnapshot[applied={T|T < 7316331236226027520}]
I20260808 18:28:00.727392 19783 tablet_replica.cc:439] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ed97518de6fe471da5fae6aa597806de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } }
I20260808 18:28:00.727492 19783 tablet_replica.cc:442] T 46a51d6c9688468b8a31f54b7f86ced4 P 892d0db548c4449c8d67d8f629d38257: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:28:00.727530 19788 tablet_replica.cc:439] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ed97518de6fe471da5fae6aa597806de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } } }
I20260808 18:28:00.727622 19788 tablet_replica.cc:442] T 46a51d6c9688468b8a31f54b7f86ced4 P 906fd6cd59e44e3db022089c7c1b14b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:28:00.735800 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.732333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48342 (local address 127.7.164.67:38335)
0808 18:28:00.732467 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:00.732475 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:00.732489 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:28:00.732610 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:00.732612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:00.732651 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:00.732712 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:28:00.732963 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.733463 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.734720 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.734928 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.735045 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.735290 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:28:00.735358 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:28:00.735413 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.735575 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.735631 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:28:00.746606 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.743114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:49091 (local address 127.7.164.67:38335)
0808 18:28:00.743244 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:00.743248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:00.743463 (+ 215us) server_negotiation.cc:407] Connection header received
0808 18:28:00.743565 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:00.743567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:00.743606 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:00.743665 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:28:00.743942 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.744386 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.745583 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.745776 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.745851 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.746168 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:28:00.746248 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:28:00.746302 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.746410 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.746466 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":57}
I20260808 18:28:00.746753 19849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.743054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:38335 (local address 127.7.164.67:49091)
0808 18:28:00.743372 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:00.743386 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:00.743489 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:00.743697 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:00.743700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:00.743707 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:28:00.743840 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.743846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.744485 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.744488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.745466 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:00.745476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.745884 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.745888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:00.745982 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.746040 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:28:00.746324 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:28:00.746331 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 18:28:00.746400 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 18:28:00.746504 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":19,"thread_start_us":128,"threads_started":1}
I20260808 18:28:00.753407 19468 raft_consensus.cc:1261] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Refusing update from remote peer ed97518de6fe471da5fae6aa597806de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:28:00.753527 19265 raft_consensus.cc:1261] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Refusing update from remote peer ed97518de6fe471da5fae6aa597806de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:28:00.753842 19821 consensus_queue.cc:1035] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [LEADER]: Connected to new peer: Peer: permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:28:00.754031 19787 consensus_queue.cc:1035] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [LEADER]: Connected to new peer: Peer: permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:28:00.796474 19044 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:28:00.796657 19044 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:28:00.798812 19044 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9: Sending DeleteTablet for 3 replicas of tablet 8b18f057d5714c2298e6515c0ee910b5
I20260808 18:28:00.799539 19233 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b18f057d5714c2298e6515c0ee910b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:00 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:56756
I20260808 18:28:00.799645 19639 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b18f057d5714c2298e6515c0ee910b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:00 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:48328
I20260808 18:28:00.799748 19813 tablet_replica.cc:330] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257: stopping tablet replica
I20260808 18:28:00.799774 19430 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b18f057d5714c2298e6515c0ee910b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:00 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:55874
I20260808 18:28:00.799854 19812 tablet_replica.cc:330] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de: stopping tablet replica
I20260808 18:28:00.799881 19813 raft_consensus.cc:2229] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:28:00.799942 19815 tablet_replica.cc:330] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5: stopping tablet replica
I20260808 18:28:00.799955 19812 raft_consensus.cc:2229] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:28:00.800009 19813 raft_consensus.cc:2258] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:28:00.800058 19815 raft_consensus.cc:2229] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:28:00.800168 19815 raft_consensus.cc:2258] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:28:00.800241 19812 raft_consensus.cc:2258] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:28:00.800984 19815 ts_tablet_manager.cc:1905] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:28:00.801045 19812 ts_tablet_manager.cc:1905] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:28:00.801147 19813 ts_tablet_manager.cc:1905] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:28:00.803903 19812 ts_tablet_manager.cc:1918] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:28:00.803943 19815 ts_tablet_manager.cc:1918] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:28:00.803978 19812 log.cc:1196] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/8b18f057d5714c2298e6515c0ee910b5
I20260808 18:28:00.803970 19813 ts_tablet_manager.cc:1918] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:28:00.804005 19815 log.cc:1196] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/8b18f057d5714c2298e6515c0ee910b5
I20260808 18:28:00.804055 19813 log.cc:1196] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/wal/wals/8b18f057d5714c2298e6515c0ee910b5
I20260808 18:28:00.804275 19815 ts_tablet_manager.cc:1939] T 8b18f057d5714c2298e6515c0ee910b5 P 906fd6cd59e44e3db022089c7c1b14b5: Deleting consensus metadata
I20260808 18:28:00.804286 19812 ts_tablet_manager.cc:1939] T 8b18f057d5714c2298e6515c0ee910b5 P ed97518de6fe471da5fae6aa597806de: Deleting consensus metadata
I20260808 18:28:00.804324 19813 ts_tablet_manager.cc:1939] T 8b18f057d5714c2298e6515c0ee910b5 P 892d0db548c4449c8d67d8f629d38257: Deleting consensus metadata
I20260808 18:28:00.804956 19027 catalog_manager.cc:4841] TS 906fd6cd59e44e3db022089c7c1b14b5 (127.7.164.66:38205): tablet 8b18f057d5714c2298e6515c0ee910b5 (table test-table [id=da9dd840cd0c40d78a203ddd59129f11]) successfully deleted
I20260808 18:28:00.805037 19031 catalog_manager.cc:4841] TS 892d0db548c4449c8d67d8f629d38257 (127.7.164.65:33565): tablet 8b18f057d5714c2298e6515c0ee910b5 (table test-table [id=da9dd840cd0c40d78a203ddd59129f11]) successfully deleted
I20260808 18:28:00.805703 19028 catalog_manager.cc:4841] TS ed97518de6fe471da5fae6aa597806de (127.7.164.67:38335): tablet 8b18f057d5714c2298e6515c0ee910b5 (table test-table [id=da9dd840cd0c40d78a203ddd59129f11]) successfully deleted
Aug 08 18:28:00 dist-test-slave-jztv krb5kdc[18983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:28:00 dist-test-slave-jztv krb5kdc[18983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:00 dist-test-slave-jztv krb5kdc[18983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 18:28:00.835078 19112 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:00.828270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46104 (local address 127.7.164.126:32975)
0808 18:28:00.828430 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:00.828435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:00.828690 (+ 255us) server_negotiation.cc:407] Connection header received
0808 18:28:00.828803 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:00.828806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:00.828871 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:00.828976 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:00.829675 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.830517 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.831370 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.831698 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.831809 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.834942 (+ 3133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46104: BlockingRecv error: recv got EOF from 127.0.0.1:46104 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 18:28:00 dist-test-slave-jztv krb5kdc[18983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM
I20260808 18:28:00.845726 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.837143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46114 (local address 127.7.164.126:32975)
0808 18:28:00.837289 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:00.837294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:00.837525 (+ 231us) server_negotiation.cc:407] Connection header received
0808 18:28:00.837654 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:00.837657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:00.837702 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:00.837801 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:00.838481 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.839002 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.839838 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.840072 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.840185 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.842710 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:00.842713 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:00.842715 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:28:00.842741 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:00.844130 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:00.844577 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:00.844580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:00.844581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:00.844659 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:00.844927 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:00.844929 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:00.844930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:00.845100 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:00.845200 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.845455 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.845554 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:28:00.848482 19044 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:28:00.848944 19044 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:28:00.854050 19430 tablet_service.cc:1467] Processing CreateTablet for tablet 972af990d90243d5a2476a6faef87883 (DEFAULT_TABLE table=test-table [id=a9059a2fccb74287b24f33cbda4758e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:00.854084 19233 tablet_service.cc:1467] Processing CreateTablet for tablet 972af990d90243d5a2476a6faef87883 (DEFAULT_TABLE table=test-table [id=a9059a2fccb74287b24f33cbda4758e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:00.854130 19639 tablet_service.cc:1467] Processing CreateTablet for tablet 972af990d90243d5a2476a6faef87883 (DEFAULT_TABLE table=test-table [id=a9059a2fccb74287b24f33cbda4758e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:00.854434 19639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 972af990d90243d5a2476a6faef87883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:00.854434 19430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 972af990d90243d5a2476a6faef87883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:00.854447 19233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 972af990d90243d5a2476a6faef87883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:00.856917 19777 tablet_bootstrap.cc:492] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de: Bootstrap starting.
I20260808 18:28:00.856917 19775 tablet_bootstrap.cc:492] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257: Bootstrap starting.
I20260808 18:28:00.857654 19775 tablet_bootstrap.cc:654] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:00.857718 19777 tablet_bootstrap.cc:654] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:00.858179 19778 tablet_bootstrap.cc:492] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5: Bootstrap starting.
I20260808 18:28:00.858779 19775 tablet_bootstrap.cc:492] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257: No bootstrap required, opened a new log
I20260808 18:28:00.858868 19775 ts_tablet_manager.cc:1397] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 18:28:00.859124 19778 tablet_bootstrap.cc:654] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:00.859406 19775 raft_consensus.cc:348] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.859543 19775 raft_consensus.cc:374] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:00.859586 19775 raft_consensus.cc:729] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892d0db548c4449c8d67d8f629d38257, State: Initialized, Role: FOLLOWER
I20260808 18:28:00.859735 19775 consensus_queue.cc:260] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.860059 19775 ts_tablet_manager.cc:1428] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 18:28:00.860456 19778 tablet_bootstrap.cc:492] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5: No bootstrap required, opened a new log
I20260808 18:28:00.860571 19778 ts_tablet_manager.cc:1397] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:28:00.860996 19778 raft_consensus.cc:348] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.861114 19778 raft_consensus.cc:374] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:00.861155 19778 raft_consensus.cc:729] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906fd6cd59e44e3db022089c7c1b14b5, State: Initialized, Role: FOLLOWER
I20260808 18:28:00.861294 19778 consensus_queue.cc:260] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.861598 19778 ts_tablet_manager.cc:1428] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:28:00.862972 19777 tablet_bootstrap.cc:492] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de: No bootstrap required, opened a new log
I20260808 18:28:00.863094 19777 ts_tablet_manager.cc:1397] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260808 18:28:00.863468 19777 raft_consensus.cc:348] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.863596 19777 raft_consensus.cc:374] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:00.863642 19777 raft_consensus.cc:729] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed97518de6fe471da5fae6aa597806de, State: Initialized, Role: FOLLOWER
I20260808 18:28:00.863775 19777 consensus_queue.cc:260] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.864065 19777 ts_tablet_manager.cc:1428] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:28:00.962944 19788 raft_consensus.cc:482] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:28:00.963145 19788 raft_consensus.cc:504] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.963661 19788 leader_election.cc:288] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 892d0db548c4449c8d67d8f629d38257 (127.7.164.65:33565), ed97518de6fe471da5fae6aa597806de (127.7.164.67:38335)
I20260808 18:28:00.964241 19668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "972af990d90243d5a2476a6faef87883" candidate_uuid: "906fd6cd59e44e3db022089c7c1b14b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed97518de6fe471da5fae6aa597806de" is_pre_election: true
I20260808 18:28:00.964242 19265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "972af990d90243d5a2476a6faef87883" candidate_uuid: "906fd6cd59e44e3db022089c7c1b14b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892d0db548c4449c8d67d8f629d38257" is_pre_election: true
I20260808 18:28:00.964416 19265 raft_consensus.cc:2454] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 906fd6cd59e44e3db022089c7c1b14b5 in term 0.
I20260808 18:28:00.964444 19668 raft_consensus.cc:2454] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 906fd6cd59e44e3db022089c7c1b14b5 in term 0.
I20260808 18:28:00.964767 19368 leader_election.cc:302] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 892d0db548c4449c8d67d8f629d38257, 906fd6cd59e44e3db022089c7c1b14b5; no voters:
I20260808 18:28:00.965006 19788 raft_consensus.cc:2781] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:28:00.965082 19788 raft_consensus.cc:482] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:28:00.965127 19788 raft_consensus.cc:3037] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:00.966549 19788 raft_consensus.cc:504] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.966962 19788 leader_election.cc:288] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [CANDIDATE]: Term 1 election: Requested vote from peers 892d0db548c4449c8d67d8f629d38257 (127.7.164.65:33565), ed97518de6fe471da5fae6aa597806de (127.7.164.67:38335)
I20260808 18:28:00.967386 19668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "972af990d90243d5a2476a6faef87883" candidate_uuid: "906fd6cd59e44e3db022089c7c1b14b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed97518de6fe471da5fae6aa597806de"
I20260808 18:28:00.967387 19265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "972af990d90243d5a2476a6faef87883" candidate_uuid: "906fd6cd59e44e3db022089c7c1b14b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "892d0db548c4449c8d67d8f629d38257"
I20260808 18:28:00.967530 19265 raft_consensus.cc:3037] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:00.967532 19668 raft_consensus.cc:3037] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:00.968479 19668 raft_consensus.cc:2454] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 906fd6cd59e44e3db022089c7c1b14b5 in term 1.
I20260808 18:28:00.968781 19265 raft_consensus.cc:2454] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 906fd6cd59e44e3db022089c7c1b14b5 in term 1.
I20260808 18:28:00.969125 19368 leader_election.cc:302] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 892d0db548c4449c8d67d8f629d38257, 906fd6cd59e44e3db022089c7c1b14b5; no voters:
I20260808 18:28:00.969434 19788 raft_consensus.cc:2781] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:28:00.969641 19788 raft_consensus.cc:686] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 LEADER]: Becoming Leader. State: Replica: 906fd6cd59e44e3db022089c7c1b14b5, State: Running, Role: LEADER
I20260808 18:28:00.970187 19788 consensus_queue.cc:237] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:00.972926 19046 catalog_manager.cc:5495] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 906fd6cd59e44e3db022089c7c1b14b5 (127.7.164.66). New cstate: current_term: 1 leader_uuid: "906fd6cd59e44e3db022089c7c1b14b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } health_report { overall_health: UNKNOWN } } }
I20260808 18:28:00.989897 19883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:00.985618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55886 (local address 127.7.164.66:38205)
0808 18:28:00.985831 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:00.985835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:00.985848 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:28:00.985922 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:00.985925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:00.985977 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:00.986090 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:28:00.986418 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.986923 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.988156 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:00.988368 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:00.988446 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:00.988779 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:28:00.989338 (+ 559us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:28:00.989431 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:00.989575 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 18:28:00.989687 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":75,"threads_started":1}
I20260808 18:28:00.992651 19668 raft_consensus.cc:1261] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Refusing update from remote peer 906fd6cd59e44e3db022089c7c1b14b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:28:00.992734 19265 raft_consensus.cc:1261] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Refusing update from remote peer 906fd6cd59e44e3db022089c7c1b14b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:28:00.993136 19788 consensus_queue.cc:1035] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:28:00.993314 19879 consensus_queue.cc:1035] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:28:01.005501 19046 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:28:01.005668 19046 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:28:01.007953 19046 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9: Sending DeleteTablet for 3 replicas of tablet 972af990d90243d5a2476a6faef87883
I20260808 18:28:01.008494 19233 tablet_service.cc:1514] Processing DeleteTablet for tablet 972af990d90243d5a2476a6faef87883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:00 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:56756
I20260808 18:28:01.008584 19430 tablet_service.cc:1514] Processing DeleteTablet for tablet 972af990d90243d5a2476a6faef87883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:00 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:55874
I20260808 18:28:01.008790 19813 tablet_replica.cc:330] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257: stopping tablet replica
I20260808 18:28:01.008791 19815 tablet_replica.cc:330] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5: stopping tablet replica
I20260808 18:28:01.008911 19815 raft_consensus.cc:2229] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:28:01.008921 19813 raft_consensus.cc:2229] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:28:01.009080 19813 raft_consensus.cc:2258] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:28:01.009213 19815 raft_consensus.cc:2258] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:28:01.010053 19813 ts_tablet_manager.cc:1905] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:28:01.010092 19815 ts_tablet_manager.cc:1905] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:28:01.010398 19639 tablet_service.cc:1514] Processing DeleteTablet for tablet 972af990d90243d5a2476a6faef87883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:00 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:48328
I20260808 18:28:01.010571 19812 tablet_replica.cc:330] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de: stopping tablet replica
I20260808 18:28:01.010672 19812 raft_consensus.cc:2229] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:28:01.010776 19812 raft_consensus.cc:2258] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:28:01.011284 19046 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:28:01.011746 19046 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:28:01.012459 19813 ts_tablet_manager.cc:1918] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:28:01.012537 19813 log.cc:1196] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-0/wal/wals/972af990d90243d5a2476a6faef87883
I20260808 18:28:01.012792 19813 ts_tablet_manager.cc:1939] T 972af990d90243d5a2476a6faef87883 P 892d0db548c4449c8d67d8f629d38257: Deleting consensus metadata
I20260808 18:28:01.012782 19815 ts_tablet_manager.cc:1918] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:28:01.012856 19815 log.cc:1196] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/972af990d90243d5a2476a6faef87883
I20260808 18:28:01.013166 19815 ts_tablet_manager.cc:1939] T 972af990d90243d5a2476a6faef87883 P 906fd6cd59e44e3db022089c7c1b14b5: Deleting consensus metadata
I20260808 18:28:01.013533 19031 catalog_manager.cc:4841] TS 892d0db548c4449c8d67d8f629d38257 (127.7.164.65:33565): tablet 972af990d90243d5a2476a6faef87883 (table test-table [id=a9059a2fccb74287b24f33cbda4758e9]) successfully deleted
I20260808 18:28:01.013834 19027 catalog_manager.cc:4841] TS 906fd6cd59e44e3db022089c7c1b14b5 (127.7.164.66:38205): tablet 972af990d90243d5a2476a6faef87883 (table test-table [id=a9059a2fccb74287b24f33cbda4758e9]) successfully deleted
I20260808 18:28:01.016947 19233 tablet_service.cc:1467] Processing CreateTablet for tablet 58fb3f6da41245499ed5902e959a2d6a (DEFAULT_TABLE table=test-table [id=0108686dda744003b700e68603a6665b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:01.017236 19233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58fb3f6da41245499ed5902e959a2d6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:01.017233 19430 tablet_service.cc:1467] Processing CreateTablet for tablet 58fb3f6da41245499ed5902e959a2d6a (DEFAULT_TABLE table=test-table [id=0108686dda744003b700e68603a6665b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:01.017453 19430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58fb3f6da41245499ed5902e959a2d6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:01.017609 19639 tablet_service.cc:1467] Processing CreateTablet for tablet 58fb3f6da41245499ed5902e959a2d6a (DEFAULT_TABLE table=test-table [id=0108686dda744003b700e68603a6665b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:01.017858 19639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58fb3f6da41245499ed5902e959a2d6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:01.019680 19778 tablet_bootstrap.cc:492] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5: Bootstrap starting.
I20260808 18:28:01.020428 19777 tablet_bootstrap.cc:492] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de: Bootstrap starting.
I20260808 18:28:01.020524 19775 tablet_bootstrap.cc:492] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257: Bootstrap starting.
I20260808 18:28:01.020552 19778 tablet_bootstrap.cc:654] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:01.021220 19777 tablet_bootstrap.cc:654] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:01.021260 19775 tablet_bootstrap.cc:654] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:01.022374 19777 tablet_bootstrap.cc:492] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de: No bootstrap required, opened a new log
I20260808 18:28:01.022468 19777 ts_tablet_manager.cc:1397] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:28:01.022957 19777 raft_consensus.cc:348] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:01.023082 19777 raft_consensus.cc:374] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:01.023125 19777 raft_consensus.cc:729] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed97518de6fe471da5fae6aa597806de, State: Initialized, Role: FOLLOWER
I20260808 18:28:01.023259 19777 consensus_queue.cc:260] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:01.023556 19777 ts_tablet_manager.cc:1428] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:28:01.023569 19812 ts_tablet_manager.cc:1905] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:28:01.023967 19778 tablet_bootstrap.cc:492] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5: No bootstrap required, opened a new log
I20260808 18:28:01.024067 19778 ts_tablet_manager.cc:1397] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 18:28:01.024492 19778 raft_consensus.cc:348] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:01.024605 19778 raft_consensus.cc:374] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:01.024637 19778 raft_consensus.cc:729] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906fd6cd59e44e3db022089c7c1b14b5, State: Initialized, Role: FOLLOWER
I20260808 18:28:01.024762 19778 consensus_queue.cc:260] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:01.025115 19778 ts_tablet_manager.cc:1428] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260808 18:28:01.026283 19812 ts_tablet_manager.cc:1918] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:28:01.026361 19812 log.cc:1196] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/972af990d90243d5a2476a6faef87883
I20260808 18:28:01.026647 19812 ts_tablet_manager.cc:1939] T 972af990d90243d5a2476a6faef87883 P ed97518de6fe471da5fae6aa597806de: Deleting consensus metadata
I20260808 18:28:01.027901 19028 catalog_manager.cc:4841] TS ed97518de6fe471da5fae6aa597806de (127.7.164.67:38335): tablet 972af990d90243d5a2476a6faef87883 (table test-table [id=a9059a2fccb74287b24f33cbda4758e9]) successfully deleted
I20260808 18:28:01.028102 19775 tablet_bootstrap.cc:492] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257: No bootstrap required, opened a new log
I20260808 18:28:01.028201 19775 ts_tablet_manager.cc:1397] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260808 18:28:01.028615 19775 raft_consensus.cc:348] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:01.028772 19775 raft_consensus.cc:374] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:01.028813 19775 raft_consensus.cc:729] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892d0db548c4449c8d67d8f629d38257, State: Initialized, Role: FOLLOWER
I20260808 18:28:01.028934 19775 consensus_queue.cc:260] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:01.029193 19775 ts_tablet_manager.cc:1428] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:28:01.145112 19783 raft_consensus.cc:482] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:28:01.145279 19783 raft_consensus.cc:504] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:01.145848 19783 leader_election.cc:288] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 906fd6cd59e44e3db022089c7c1b14b5 (127.7.164.66:38205), ed97518de6fe471da5fae6aa597806de (127.7.164.67:38335)
I20260808 18:28:01.146303 19468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58fb3f6da41245499ed5902e959a2d6a" candidate_uuid: "892d0db548c4449c8d67d8f629d38257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906fd6cd59e44e3db022089c7c1b14b5" is_pre_election: true
I20260808 18:28:01.146476 19468 raft_consensus.cc:2454] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 892d0db548c4449c8d67d8f629d38257 in term 0.
I20260808 18:28:01.146517 19668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58fb3f6da41245499ed5902e959a2d6a" candidate_uuid: "892d0db548c4449c8d67d8f629d38257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed97518de6fe471da5fae6aa597806de" is_pre_election: true
I20260808 18:28:01.146725 19668 raft_consensus.cc:2454] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 892d0db548c4449c8d67d8f629d38257 in term 0.
I20260808 18:28:01.146816 19166 leader_election.cc:302] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 892d0db548c4449c8d67d8f629d38257, 906fd6cd59e44e3db022089c7c1b14b5; no voters:
I20260808 18:28:01.147013 19783 raft_consensus.cc:2781] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:28:01.147070 19783 raft_consensus.cc:482] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:28:01.147120 19783 raft_consensus.cc:3037] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:01.148125 19783 raft_consensus.cc:504] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:01.148489 19783 leader_election.cc:288] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [CANDIDATE]: Term 1 election: Requested vote from peers 906fd6cd59e44e3db022089c7c1b14b5 (127.7.164.66:38205), ed97518de6fe471da5fae6aa597806de (127.7.164.67:38335)
I20260808 18:28:01.148869 19668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58fb3f6da41245499ed5902e959a2d6a" candidate_uuid: "892d0db548c4449c8d67d8f629d38257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed97518de6fe471da5fae6aa597806de"
I20260808 18:28:01.148887 19468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58fb3f6da41245499ed5902e959a2d6a" candidate_uuid: "892d0db548c4449c8d67d8f629d38257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906fd6cd59e44e3db022089c7c1b14b5"
I20260808 18:28:01.149006 19468 raft_consensus.cc:3037] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:01.149009 19668 raft_consensus.cc:3037] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:01.149976 19668 raft_consensus.cc:2454] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 892d0db548c4449c8d67d8f629d38257 in term 1.
I20260808 18:28:01.150321 19167 leader_election.cc:302] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 892d0db548c4449c8d67d8f629d38257, ed97518de6fe471da5fae6aa597806de; no voters:
I20260808 18:28:01.150342 19468 raft_consensus.cc:2454] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 892d0db548c4449c8d67d8f629d38257 in term 1.
I20260808 18:28:01.150542 19783 raft_consensus.cc:2781] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:28:01.150705 19783 raft_consensus.cc:686] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [term 1 LEADER]: Becoming Leader. State: Replica: 892d0db548c4449c8d67d8f629d38257, State: Running, Role: LEADER
I20260808 18:28:01.150852 19783 consensus_queue.cc:237] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } }
I20260808 18:28:01.152714 19046 catalog_manager.cc:5495] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 reported cstate change: term changed from 0 to 1, leader changed from <none> to 892d0db548c4449c8d67d8f629d38257 (127.7.164.65). New cstate: current_term: 1 leader_uuid: "892d0db548c4449c8d67d8f629d38257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "892d0db548c4449c8d67d8f629d38257" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 } health_report { overall_health: UNKNOWN } } }
I20260808 18:28:01.187500 19894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:01.184436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56784 (local address 127.7.164.65:33565)
0808 18:28:01.184663 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:01.184666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:01.184678 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:28:01.184718 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:01.184720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:01.184761 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:01.184836 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:28:01.185056 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:01.185631 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:01.186480 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:01.186793 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:01.186885 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:01.187120 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:28:01.187189 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:28:01.187244 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:01.187335 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 18:28:01.187381 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260808 18:28:01.197242 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:01.194410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:43013 (local address 127.7.164.67:38335)
0808 18:28:01.194556 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:01.194559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:01.194673 (+ 114us) server_negotiation.cc:407] Connection header received
0808 18:28:01.194766 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:01.194768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:01.194811 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:01.194875 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:28:01.195085 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:01.195579 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:01.196355 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:01.196565 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:01.196641 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:01.196886 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:28:01.196952 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:28:01.197005 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:01.197090 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 18:28:01.197131 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:28:01.197260 19900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:01.194295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:38335 (local address 127.7.164.65:43013)
0808 18:28:01.194597 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:01.194610 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:01.194697 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:01.194901 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:01.194903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:01.194909 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:28:01.194998 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:01.195002 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:01.195672 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:01.195674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:01.196262 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:01.196269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:01.196664 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:01.196667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:01.196729 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:01.196773 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:28:01.197029 (+ 256us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:28:01.197034 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 18:28:01.197082 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 18:28:01.197148 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":101,"threads_started":1}
I20260808 18:28:01.203025 19668 raft_consensus.cc:1261] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Refusing update from remote peer 892d0db548c4449c8d67d8f629d38257: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:28:01.203411 19468 raft_consensus.cc:1261] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Refusing update from remote peer 892d0db548c4449c8d67d8f629d38257: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:28:01.203444 19783 consensus_queue.cc:1035] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed97518de6fe471da5fae6aa597806de" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:28:01.203861 19783 consensus_queue.cc:1035] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "906fd6cd59e44e3db022089c7c1b14b5" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 38205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:28:01.222261 19894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:01.219007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:51611 (local address 127.7.164.65:33565)
0808 18:28:01.219134 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:01.219138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:01.219151 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:28:01.219233 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:01.219236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:01.219280 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:01.219385 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:28:01.219625 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:01.220143 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:01.220975 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:01.221279 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:01.221376 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:01.221625 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:28:01.221708 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:28:01.221780 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:01.222032 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 18:28:01.222096 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:28:01.222317 19849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:01.218785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:33565 (local address 127.7.164.67:51611)
0808 18:28:01.219029 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:01.219064 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:01.219145 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:01.219395 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:01.219398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:01.219406 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:28:01.219513 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:01.219519 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:01.220254 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:01.220259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:01.220849 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:01.220858 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:01.221375 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:01.221379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:01.221466 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:01.221506 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:28:01.221917 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:28:01.221926 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 18:28:01.221999 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 18:28:01.222125 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260808 18:28:01.243508 19044 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:28:01.243692 19044 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:28:01.245719 19044 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 92a8443796184dafa26ae109ebb627a9: Sending DeleteTablet for 3 replicas of tablet 58fb3f6da41245499ed5902e959a2d6a
I20260808 18:28:01.246304 19233 tablet_service.cc:1514] Processing DeleteTablet for tablet 58fb3f6da41245499ed5902e959a2d6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:01 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:56756
I20260808 18:28:01.246428 19430 tablet_service.cc:1514] Processing DeleteTablet for tablet 58fb3f6da41245499ed5902e959a2d6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:01 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:55874
I20260808 18:28:01.246517 19813 tablet_replica.cc:330] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257: stopping tablet replica
I20260808 18:28:01.246492 19639 tablet_service.cc:1514] Processing DeleteTablet for tablet 58fb3f6da41245499ed5902e959a2d6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:01 UTC) from {username='oryx', principal='oryx/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:48328
I20260808 18:28:01.246606 19815 tablet_replica.cc:330] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5: stopping tablet replica
I20260808 18:28:01.246654 19813 raft_consensus.cc:2229] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:28:01.246723 19815 raft_consensus.cc:2229] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:28:01.246845 19815 raft_consensus.cc:2258] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:28:01.246862 19813 raft_consensus.cc:2258] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:28:01.247043 19812 tablet_replica.cc:330] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de: stopping tablet replica
I20260808 18:28:01.247182 19812 raft_consensus.cc:2229] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:28:01.247319 19812 raft_consensus.cc:2258] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:28:01.247473 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 19142
I20260808 18:28:01.247720 19813 ts_tablet_manager.cc:1905] T 58fb3f6da41245499ed5902e959a2d6a P 892d0db548c4449c8d67d8f629d38257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:28:01.253867 19812 ts_tablet_manager.cc:1905] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:28:01.253872 19815 ts_tablet_manager.cc:1905] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:28:01.256911 19815 ts_tablet_manager.cc:1918] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:28:01.257026 19815 log.cc:1196] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/58fb3f6da41245499ed5902e959a2d6a
I20260808 18:28:01.257270 19815 ts_tablet_manager.cc:1939] T 58fb3f6da41245499ed5902e959a2d6a P 906fd6cd59e44e3db022089c7c1b14b5: Deleting consensus metadata
I20260808 18:28:01.257355 19812 ts_tablet_manager.cc:1918] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:28:01.257448 19812 log.cc:1196] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/58fb3f6da41245499ed5902e959a2d6a
I20260808 18:28:01.257716 19812 ts_tablet_manager.cc:1939] T 58fb3f6da41245499ed5902e959a2d6a P ed97518de6fe471da5fae6aa597806de: Deleting consensus metadata
I20260808 18:28:01.258108 19027 catalog_manager.cc:4841] TS 906fd6cd59e44e3db022089c7c1b14b5 (127.7.164.66:38205): tablet 58fb3f6da41245499ed5902e959a2d6a (table test-table [id=0108686dda744003b700e68603a6665b]) successfully deleted
I20260808 18:28:01.258533 19028 catalog_manager.cc:4841] TS ed97518de6fe471da5fae6aa597806de (127.7.164.67:38335): tablet 58fb3f6da41245499ed5902e959a2d6a (table test-table [id=0108686dda744003b700e68603a6665b]) successfully deleted
W20260808 18:28:01.264415 19031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:33565 (error 108)
I20260808 18:28:01.264636 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 19342
I20260808 18:28:01.268102 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:01.266978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:33565 (local address 127.0.0.1:56798)
0808 18:28:01.267852 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:01.267945 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":764,"thread_start_us":476,"threads_started":1}
W20260808 18:28:01.268222 19031 catalog_manager.cc:4584] TS 892d0db548c4449c8d67d8f629d38257 (127.7.164.65:33565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 58fb3f6da41245499ed5902e959a2d6a: Network error: Client connection negotiation failed: client connection to 127.7.164.65:33565: connect: Connection refused (error 111)
I20260808 18:28:01.280949 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 19540
I20260808 18:28:01.292817 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 19002
2026-08-08T18:28:01Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3772 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:28:01 dist-test-slave-jztv krb5kdc[19911](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:28:01 dist-test-slave-jztv krb5kdc[19911](info): set up 2 sockets
Aug 08 18:28:01 dist-test-slave-jztv krb5kdc[19911](info): commencing operation
krb5kdc: starting...
W20260808 18:28:02.530355 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:28:02 dist-test-slave-jztv krb5kdc[19911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:28:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:28:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:28:02.669870 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:45797
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:45797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:28:02.785966 19928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:02.786347 19928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:02.786415 19928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:02.790474 19928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:28:02.790542 19928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:02.790563 19928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:02.790581 19928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:28:02.790596 19928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:28:02.796129 19928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:45797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:45797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:02.797287 19928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:02.798386 19928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:02.807327 19936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:02.807423 19928 server_base.cc:1034] running on GCE node
W20260808 18:28:02.807330 19934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:02.807332 19933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:02.808167 19928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:02.809253 19928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:02.810422 19928 hybrid_clock.cc:648] HybridClock initialized: now 1786213682810407 us; error 26 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T18:28:02Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1518 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:28:02 dist-test-slave-jztv krb5kdc[19946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:28:02 dist-test-slave-jztv krb5kdc[19946](info): set up 2 sockets
Aug 08 18:28:02 dist-test-slave-jztv krb5kdc[19946](info): commencing operation
krb5kdc: starting...
W20260808 18:28:04.052284 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:28:04 dist-test-slave-jztv krb5kdc[19946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:28:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:28:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:28:04.188772 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:33119
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:45609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:33119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:28:04.304224 19965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:04.304553 19965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:04.304662 19965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:04.308753 19965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:28:04.308838 19965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:04.308877 19965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:04.308933 19965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:28:04.308971 19965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:28:04.314363 19965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:45609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:33119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:33119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:04.315476 19965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:04.316788 19965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:04.325732 19974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:04.325800 19978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:04.325765 19973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:04.326141 19965 server_base.cc:1034] running on GCE node
I20260808 18:28:04.326763 19965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:04.327876 19965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:04.329035 19965 hybrid_clock.cc:648] HybridClock initialized: now 1786213684329007 us; error 42 us; skew 500 ppm
Aug 08 18:28:04 dist-test-slave-jztv krb5kdc[19946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213684, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:28:04.335600 19965 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:28:04.336555 19965 webserver.cc:460] Webserver started at http://127.7.164.126:39909/ using document root <none> and password file <none>
I20260808 18:28:04.337142 19965 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:04.337204 19965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:04.337435 19965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:04.339628 19965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "75b30c48a61f4027bf12bd932c40ba4f"
format_stamp: "Formatted at 2026-08-08 18:28:04 on dist-test-slave-jztv"
server_key: "2a02e9d87a304d5c7a8a1e16c0b6189d"
server_key_iv: "4c06f9ff86d7dc209213ec3de3cb3ba8"
server_key_version: "encryptionkey@0"
I20260808 18:28:04.340230 19965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "75b30c48a61f4027bf12bd932c40ba4f"
format_stamp: "Formatted at 2026-08-08 18:28:04 on dist-test-slave-jztv"
server_key: "2a02e9d87a304d5c7a8a1e16c0b6189d"
server_key_iv: "4c06f9ff86d7dc209213ec3de3cb3ba8"
server_key_version: "encryptionkey@0"
I20260808 18:28:04.344086 19965 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:28:04.346611 19989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:04.347841 19965 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:28:04.347990 19965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "75b30c48a61f4027bf12bd932c40ba4f"
format_stamp: "Formatted at 2026-08-08 18:28:04 on dist-test-slave-jztv"
server_key: "2a02e9d87a304d5c7a8a1e16c0b6189d"
server_key_iv: "4c06f9ff86d7dc209213ec3de3cb3ba8"
server_key_version: "encryptionkey@0"
I20260808 18:28:04.348105 19965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:04.375624 19965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:04.378584 19965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:04.378800 19965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:04.386401 19965 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:33119
I20260808 18:28:04.386463 20063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:33119 every 8 connection(s)
I20260808 18:28:04.387441 19965 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:28:04.390228 20064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:04.395498 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 19965
I20260808 18:28:04.395653 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:28:04.395654 20064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f: Bootstrap starting.
I20260808 18:28:04.395939 7825 external_mini_cluster.cc:1428] Setting key 0028c3f2501a677650a0343cea9c32b7
I20260808 18:28:04.398136 20064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:04.398893 20064 log.cc:824] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:28:04.400712 20064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f: No bootstrap required, opened a new log
I20260808 18:28:04.404479 20064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b30c48a61f4027bf12bd932c40ba4f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33119 } }
Aug 08 18:28:04 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:28:04.404776 20064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:04.404830 20064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b30c48a61f4027bf12bd932c40ba4f, State: Initialized, Role: FOLLOWER
I20260808 18:28:04.405323 20064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b30c48a61f4027bf12bd932c40ba4f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33119 } }
I20260808 18:28:04.405455 20064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:28:04.405541 20064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:28:04.405643 20064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:04.406592 20064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b30c48a61f4027bf12bd932c40ba4f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33119 } }
I20260808 18:28:04.406929 20064 leader_election.cc:302] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75b30c48a61f4027bf12bd932c40ba4f; no voters:
I20260808 18:28:04.407248 20064 leader_election.cc:288] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:28:04.407369 20071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:28:04.407634 20071 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 1 LEADER]: Becoming Leader. State: Replica: 75b30c48a61f4027bf12bd932c40ba4f, State: Running, Role: LEADER
I20260808 18:28:04.408032 20071 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b30c48a61f4027bf12bd932c40ba4f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33119 } }
I20260808 18:28:04.408474 20064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:28:04.409796 20074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75b30c48a61f4027bf12bd932c40ba4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b30c48a61f4027bf12bd932c40ba4f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33119 } } }
I20260808 18:28:04.409931 20074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [sys.catalog]: This master's current role is: LEADER
I20260808 18:28:04.410286 20075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75b30c48a61f4027bf12bd932c40ba4f. Latest consensus state: current_term: 1 leader_uuid: "75b30c48a61f4027bf12bd932c40ba4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b30c48a61f4027bf12bd932c40ba4f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33119 } } }
I20260808 18:28:04.410454 20075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [sys.catalog]: This master's current role is: LEADER
I20260808 18:28:04.411720 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:04.397323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42620 (local address 127.7.164.126:33119)
0808 18:28:04.397748 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:04.397765 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:04.397799 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:28:04.398523 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:04.398542 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:04.398858 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:04.399230 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:04.400063 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:04.401069 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:04.401942 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:04.402496 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:04.402671 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:04.405381 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:04.405386 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:04.405399 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:28:04.405450 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:04.407638 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:04.408198 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:04.408203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:04.408209 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:04.408343 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:04.408725 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:04.408729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:04.408731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:04.409246 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:04.409424 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:04.410423 (+ 999us) server_negotiation.cc:299] Negotiation successful
0808 18:28:04.410745 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":155,"threads_started":1}
W20260808 18:28:04.413856 20090 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:28:04.413939 20090 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:28:04.414292 20084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:28:04.415014 20084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:28:04.420987 20084 catalog_manager.cc:1347] Generated new cluster ID: 68e6fa352ee4425a841e48091d7dfdec
I20260808 18:28:04.421058 20084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:28:04.429770 20084 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:28:04.430828 20084 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:28:04.444610 20084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f: Generated new TSK 0
I20260808 18:28:04.445351 20084 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:28:04.483736 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:33119
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:45609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:28:04.616302 20100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:04.616562 20100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:04.616631 20100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:04.620607 20100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:04.620659 20100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:04.620751 20100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:28:04.625976 20100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:45609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:33119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:04.627058 20100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:04.628067 20100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:04.637867 20107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:04.637887 20108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:04.637866 20112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:04.638150 20100 server_base.cc:1034] running on GCE node
I20260808 18:28:04.638741 20100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:04.639444 20100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:04.640633 20100 hybrid_clock.cc:648] HybridClock initialized: now 1786213684640595 us; error 48 us; skew 500 ppm
Aug 08 18:28:04 dist-test-slave-jztv krb5kdc[19946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213684, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:28:04.643590 20100 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:28:04.644531 20100 webserver.cc:460] Webserver started at http://127.7.164.65:40661/ using document root <none> and password file <none>
I20260808 18:28:04.645107 20100 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:04.645198 20100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:04.645450 20100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:04.647615 20100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "251828446911498ab52424524397a388"
format_stamp: "Formatted at 2026-08-08 18:28:04 on dist-test-slave-jztv"
server_key: "eee04f86c4a96706781d8c3f6a12b3d9"
server_key_iv: "fa2ba66ee4eec778bde208cd60725ba0"
server_key_version: "encryptionkey@0"
I20260808 18:28:04.648219 20100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "251828446911498ab52424524397a388"
format_stamp: "Formatted at 2026-08-08 18:28:04 on dist-test-slave-jztv"
server_key: "eee04f86c4a96706781d8c3f6a12b3d9"
server_key_iv: "fa2ba66ee4eec778bde208cd60725ba0"
server_key_version: "encryptionkey@0"
I20260808 18:28:04.652295 20100 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:28:04.654923 20123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:04.656103 20100 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:28:04.656246 20100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "251828446911498ab52424524397a388"
format_stamp: "Formatted at 2026-08-08 18:28:04 on dist-test-slave-jztv"
server_key: "eee04f86c4a96706781d8c3f6a12b3d9"
server_key_iv: "fa2ba66ee4eec778bde208cd60725ba0"
server_key_version: "encryptionkey@0"
I20260808 18:28:04.656364 20100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:04.670387 20100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:04.673403 20100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:04.673620 20100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:04.674242 20100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:04.675119 20100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:04.675192 20100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:04.675266 20100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:04.675316 20100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:04.684749 20100 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:36073
I20260808 18:28:04.684768 20295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:36073 every 8 connection(s)
I20260808 18:28:04.685717 20100 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:28:04.692454 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 20100
I20260808 18:28:04.692619 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:28:04.692904 7825 external_mini_cluster.cc:1428] Setting key c4ca65acee834d2c5237a615403899f3
Aug 08 18:28:04 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213684, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:28:04.700673 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:04.687757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:59061 (local address 127.7.164.126:33119)
0808 18:28:04.687917 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:04.687921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:04.688836 (+ 915us) server_negotiation.cc:407] Connection header received
0808 18:28:04.690112 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:04.690115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:04.690165 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:04.690301 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:04.691499 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:04.692041 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:04.693375 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:04.693636 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:04.693744 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:04.696745 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:04.696750 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:04.696752 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:28:04.696784 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:04.698425 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:04.699000 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:04.699004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:04.699006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:04.699105 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:04.699515 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:04.699519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:04.699521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:04.699721 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:04.699837 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:04.700365 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 18:28:04.700505 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:28:04.701658 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:04.688205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:59061)
0808 18:28:04.688668 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:04.688713 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:04.689905 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:04.690467 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:04.690474 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:04.690856 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:04.691347 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:04.691362 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:04.692153 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:04.692155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:04.693229 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:04.693240 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:04.693788 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:04.693792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:04.693947 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:04.694699 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:04.694722 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:04.696526 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:28:04.698550 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:28:04.698555 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:28:04.698570 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:28:04.698884 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:28:04.699203 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:28:04.699209 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:28:04.699212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:28:04.699400 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:28:04.699836 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:28:04.699843 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:28:04.700116 (+ 273us) client_negotiation.cc:769] Sending connection context
0808 18:28:04.700316 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 18:28:04.700672 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260808 18:28:04.703024 20296 heartbeater.cc:347] Connected to a master server at 127.7.164.126:33119
I20260808 18:28:04.703305 20296 heartbeater.cc:464] Registering TS with master...
I20260808 18:28:04.703974 20296 heartbeater.cc:511] Master 127.7.164.126:33119 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:28:04.705775 20013 ts_manager.cc:194] Registered new tserver with Master: 251828446911498ab52424524397a388 (127.7.164.65:36073)
I20260808 18:28:04.707067 20013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:59061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:28:04.725464 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:33119
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:45609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:28:04.839748 20302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:04.840009 20302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:04.840088 20302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:04.844125 20302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:04.844177 20302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:04.844281 20302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:28:04.849462 20302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:45609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:33119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:04.850564 20302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:04.851670 20302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:04.861654 20308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:04.861790 20309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:04.861793 20311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:04.862062 20302 server_base.cc:1034] running on GCE node
I20260808 18:28:04.862511 20302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:04.863149 20302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:04.864341 20302 hybrid_clock.cc:648] HybridClock initialized: now 1786213684864318 us; error 36 us; skew 500 ppm
Aug 08 18:28:04 dist-test-slave-jztv krb5kdc[19946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213684, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:28:04.867146 20302 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:28:04.867996 20302 webserver.cc:460] Webserver started at http://127.7.164.66:40773/ using document root <none> and password file <none>
I20260808 18:28:04.868584 20302 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:04.868680 20302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:04.868919 20302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:04.871348 20302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "0fdf7d0694a2460bb0d9d0177ef70469"
format_stamp: "Formatted at 2026-08-08 18:28:04 on dist-test-slave-jztv"
server_key: "25738156a78b7f23cd1518d3761d72ad"
server_key_iv: "949324f1e46fd65382347aa77f592c48"
server_key_version: "encryptionkey@0"
I20260808 18:28:04.871968 20302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "0fdf7d0694a2460bb0d9d0177ef70469"
format_stamp: "Formatted at 2026-08-08 18:28:04 on dist-test-slave-jztv"
server_key: "25738156a78b7f23cd1518d3761d72ad"
server_key_iv: "949324f1e46fd65382347aa77f592c48"
server_key_version: "encryptionkey@0"
I20260808 18:28:04.875767 20302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:28:04.878365 20326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:04.879607 20302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:28:04.879750 20302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "0fdf7d0694a2460bb0d9d0177ef70469"
format_stamp: "Formatted at 2026-08-08 18:28:04 on dist-test-slave-jztv"
server_key: "25738156a78b7f23cd1518d3761d72ad"
server_key_iv: "949324f1e46fd65382347aa77f592c48"
server_key_version: "encryptionkey@0"
I20260808 18:28:04.879869 20302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:04.895256 20302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:04.898473 20302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:04.898682 20302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:04.899312 20302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:04.900207 20302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:04.900295 20302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:04.900390 20302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:04.900444 20302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:04.910041 20302 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:36703
I20260808 18:28:04.910084 20483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:36703 every 8 connection(s)
I20260808 18:28:04.911326 20302 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:28:04.912322 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 20302
I20260808 18:28:04.912410 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:28:04.912761 7825 external_mini_cluster.cc:1428] Setting key 0f59ab7c8da15509e73f32f95c375887
Aug 08 18:28:04 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213684, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
I20260808 18:28:04.926574 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:04.914314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:53169 (local address 127.7.164.126:33119)
0808 18:28:04.914430 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:04.914435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:04.914449 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:28:04.916386 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:04.916388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:04.916435 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:04.916571 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:04.917783 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:04.918437 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:04.919573 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:04.919990 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:04.920132 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:04.923078 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:04.923081 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:04.923083 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:28:04.923107 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:04.924630 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:04.925111 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:04.925113 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:04.925114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:04.925189 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:04.925514 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:04.925517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:04.925518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:04.925693 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:04.925767 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:04.926318 (+ 551us) server_negotiation.cc:299] Negotiation successful
0808 18:28:04.926439 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
I20260808 18:28:04.927591 20487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:04.913735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:53169)
0808 18:28:04.914288 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:04.914349 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:04.916155 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:04.916771 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:04.916778 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:04.917152 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:04.917635 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:04.917648 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:04.918553 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:04.918556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:04.919444 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:04.919453 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:04.920162 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:04.920188 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:04.920386 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:04.921019 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:04.921036 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:04.922895 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:28:04.924742 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:28:04.924747 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:28:04.924759 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:28:04.925021 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:28:04.925283 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:28:04.925289 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:28:04.925293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:28:04.925440 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:28:04.925783 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:28:04.925788 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:28:04.926038 (+ 250us) client_negotiation.cc:769] Sending connection context
0808 18:28:04.926198 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 18:28:04.926606 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":121,"threads_started":1}
I20260808 18:28:04.928838 20485 heartbeater.cc:347] Connected to a master server at 127.7.164.126:33119
I20260808 18:28:04.929126 20485 heartbeater.cc:464] Registering TS with master...
I20260808 18:28:04.929675 20485 heartbeater.cc:511] Master 127.7.164.126:33119 requested a full tablet report, sending...
I20260808 18:28:04.930833 20013 ts_manager.cc:194] Registered new tserver with Master: 0fdf7d0694a2460bb0d9d0177ef70469 (127.7.164.66:36703)
I20260808 18:28:04.931420 20013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:53169
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:28:04.946314 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:33119
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:45609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:28:05.061113 20491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:05.061371 20491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:05.061414 20491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:05.065506 20491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:05.065573 20491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:05.065634 20491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:28:05.070978 20491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:45609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:33119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:05.072000 20491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:05.073045 20491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:05.082585 20501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:05.082585 20498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:05.082605 20497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:05.083200 20491 server_base.cc:1034] running on GCE node
I20260808 18:28:05.083640 20491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:05.084410 20491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:05.085593 20491 hybrid_clock.cc:648] HybridClock initialized: now 1786213685085574 us; error 28 us; skew 500 ppm
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213685, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:28:05.088558 20491 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:28:05.089466 20491 webserver.cc:460] Webserver started at http://127.7.164.67:40267/ using document root <none> and password file <none>
I20260808 18:28:05.090055 20491 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:05.090121 20491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:05.090399 20491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:05.092540 20491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "ee030f68bc924c77804f850639917a87"
format_stamp: "Formatted at 2026-08-08 18:28:05 on dist-test-slave-jztv"
server_key: "2806c42386a7a560c47e1f8fcb79c298"
server_key_iv: "c1707ceb3067ce7c1e29337fb330b09a"
server_key_version: "encryptionkey@0"
I20260808 18:28:05.093158 20491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "ee030f68bc924c77804f850639917a87"
format_stamp: "Formatted at 2026-08-08 18:28:05 on dist-test-slave-jztv"
server_key: "2806c42386a7a560c47e1f8fcb79c298"
server_key_iv: "c1707ceb3067ce7c1e29337fb330b09a"
server_key_version: "encryptionkey@0"
I20260808 18:28:05.097069 20491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 18:28:05.099716 20510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:05.100961 20491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:28:05.101071 20491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "ee030f68bc924c77804f850639917a87"
format_stamp: "Formatted at 2026-08-08 18:28:05 on dist-test-slave-jztv"
server_key: "2806c42386a7a560c47e1f8fcb79c298"
server_key_iv: "c1707ceb3067ce7c1e29337fb330b09a"
server_key_version: "encryptionkey@0"
I20260808 18:28:05.101204 20491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:05.118340 20491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:05.121459 20491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:05.121685 20491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:05.122383 20491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:05.123248 20491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:05.123296 20491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:05.123368 20491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:05.123400 20491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:05.132727 20491 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:43677
I20260808 18:28:05.132754 20679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:43677 every 8 connection(s)
I20260808 18:28:05.133641 20491 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:28:05.133917 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 20491
I20260808 18:28:05.134071 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:28:05.134511 7825 external_mini_cluster.cc:1428] Setting key 022cee09ac8d8f4aee5435a5e153e8b2
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213685, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:28:05.147612 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:05.135618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:51835 (local address 127.7.164.126:33119)
0808 18:28:05.135755 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:05.135758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:05.136826 (+ 1068us) server_negotiation.cc:407] Connection header received
0808 18:28:05.138361 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:05.138363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:05.138411 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:05.138479 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:05.139610 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.140129 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.141216 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.141425 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.141512 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:05.143942 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:05.143945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:05.143947 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:28:05.143972 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:05.145454 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:05.145946 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:05.145949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:05.145951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:05.146028 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:05.146408 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:05.146410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:05.146412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:05.146608 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:05.146687 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:05.147254 (+ 567us) server_negotiation.cc:299] Negotiation successful
0808 18:28:05.147398 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:28:05.148612 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:05.136180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:51835)
0808 18:28:05.136670 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:05.136723 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:05.138083 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:05.138621 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:05.138627 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:05.138971 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:05.139472 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:05.139486 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.140235 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.140238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:05.141107 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:05.141116 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.141572 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.141575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:05.141727 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:05.142328 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:05.142343 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:05.143787 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:28:05.145581 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:28:05.145586 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:28:05.145598 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:28:05.145860 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:28:05.146111 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:28:05.146113 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:28:05.146115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:28:05.146317 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:28:05.146701 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:28:05.146707 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:28:05.146939 (+ 232us) client_negotiation.cc:769] Sending connection context
0808 18:28:05.147167 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 18:28:05.147510 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":142,"threads_started":1}
I20260808 18:28:05.149914 20680 heartbeater.cc:347] Connected to a master server at 127.7.164.126:33119
I20260808 18:28:05.150179 20680 heartbeater.cc:464] Registering TS with master...
I20260808 18:28:05.150785 20680 heartbeater.cc:511] Master 127.7.164.126:33119 requested a full tablet report, sending...
I20260808 18:28:05.151872 20013 ts_manager.cc:194] Registered new tserver with Master: ee030f68bc924c77804f850639917a87 (127.7.164.67:43677)
I20260808 18:28:05.152460 20013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:51835
I20260808 18:28:05.163126 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.65.keytab.
Entry for principal oryx/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.65.keytab.
WARNING: no policy specified for oryx/127.7.164.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.164.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.66.keytab.
Entry for principal oryx/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.66.keytab.
WARNING: no policy specified for oryx/127.7.164.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.164.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.67.keytab.
Entry for principal oryx/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.67.keytab.
I20260808 18:28:05.244657 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 20100
I20260808 18:28:05.261483 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 20302
I20260808 18:28:05.271047 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 20491
I20260808 18:28:05.282972 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 19965
I20260808 18:28:05.295451 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:33119
--webserver_interface=127.7.164.126
--webserver_port=39909
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:45609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:33119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:28:05.408219 20694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:05.408499 20694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:05.408581 20694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:05.412586 20694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:28:05.412643 20694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:05.412663 20694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:05.412680 20694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:28:05.412696 20694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:28:05.417966 20694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:45609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:33119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:33119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:05.419095 20694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:05.420189 20694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:05.428942 20702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:05.428920 20701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:05.428925 20705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:05.429301 20694 server_base.cc:1034] running on GCE node
I20260808 18:28:05.429881 20694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:05.431049 20694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:05.432242 20694 hybrid_clock.cc:648] HybridClock initialized: now 1786213685432208 us; error 48 us; skew 500 ppm
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213685, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:28:05.435178 20694 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:28:05.436134 20694 webserver.cc:460] Webserver started at http://127.7.164.126:39909/ using document root <none> and password file <none>
I20260808 18:28:05.436713 20694 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:05.436832 20694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:05.440784 20694 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:28:05.442999 20716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:05.444235 20694 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:28:05.444387 20694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "75b30c48a61f4027bf12bd932c40ba4f"
format_stamp: "Formatted at 2026-08-08 18:28:04 on dist-test-slave-jztv"
server_key: "2a02e9d87a304d5c7a8a1e16c0b6189d"
server_key_iv: "4c06f9ff86d7dc209213ec3de3cb3ba8"
server_key_version: "encryptionkey@0"
I20260808 18:28:05.444953 20694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:05.458760 20694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:05.461602 20694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:05.461817 20694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:05.469811 20694 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:33119
I20260808 18:28:05.469838 20803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:33119 every 8 connection(s)
I20260808 18:28:05.470924 20694 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:28:05.472326 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 20694
I20260808 18:28:05.472857 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:36073
--local_ip_for_outbound_sockets=127.7.164.65
--tserver_master_addrs=127.7.164.126:33119
--webserver_port=40661
--webserver_interface=127.7.164.65
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:45609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.164.65
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.65.keytab with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:28:05.476219 20804 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:28:05.477980 20804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f: Bootstrap starting.
I20260808 18:28:05.495040 20804 log.cc:824] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:28:05.501500 20804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:28:05.501900 20804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f: Bootstrap complete.
I20260808 18:28:05.506111 20804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b30c48a61f4027bf12bd932c40ba4f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33119 } }
I20260808 18:28:05.506601 20804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b30c48a61f4027bf12bd932c40ba4f, State: Initialized, Role: FOLLOWER
I20260808 18:28:05.507126 20804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b30c48a61f4027bf12bd932c40ba4f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33119 } }
I20260808 18:28:05.507243 20804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:28:05.507331 20804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:28:05.507455 20804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:28:05.509359 20804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b30c48a61f4027bf12bd932c40ba4f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33119 } }
I20260808 18:28:05.509698 20804 leader_election.cc:302] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75b30c48a61f4027bf12bd932c40ba4f; no voters:
I20260808 18:28:05.510004 20804 leader_election.cc:288] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 18:28:05.510139 20808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 2 FOLLOWER]: Leader election won for term 2
I20260808 18:28:05.510414 20808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [term 2 LEADER]: Becoming Leader. State: Replica: 75b30c48a61f4027bf12bd932c40ba4f, State: Running, Role: LEADER
I20260808 18:28:05.510721 20808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b30c48a61f4027bf12bd932c40ba4f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33119 } }
I20260808 18:28:05.511238 20804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:28:05.512290 20809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "75b30c48a61f4027bf12bd932c40ba4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b30c48a61f4027bf12bd932c40ba4f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33119 } } }
I20260808 18:28:05.512301 20812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75b30c48a61f4027bf12bd932c40ba4f. Latest consensus state: current_term: 2 leader_uuid: "75b30c48a61f4027bf12bd932c40ba4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b30c48a61f4027bf12bd932c40ba4f" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33119 } } }
I20260808 18:28:05.512418 20809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [sys.catalog]: This master's current role is: LEADER
I20260808 18:28:05.512431 20812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f [sys.catalog]: This master's current role is: LEADER
I20260808 18:28:05.512764 20816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:28:05.515982 20816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:28:05.516619 20816 catalog_manager.cc:1259] Loaded cluster ID: 68e6fa352ee4425a841e48091d7dfdec
I20260808 18:28:05.516686 20816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:28:05.519229 20816 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:28:05.520076 20816 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 75b30c48a61f4027bf12bd932c40ba4f: Loaded TSK: 0
I20260808 18:28:05.521296 20816 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:28:05.591531 20806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:05.591804 20806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:05.591849 20806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:05.596004 20806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:05.596073 20806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:05.596136 20806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:28:05.601624 20806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:45609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.65.keytab
--principal=oryx/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:36073
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=40661
--tserver_master_addrs=127.7.164.126:33119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:05.602838 20806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:05.603919 20806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:05.613994 20839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:05.614140 20837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:05.613986 20843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:05.614463 20806 server_base.cc:1034] running on GCE node
I20260808 18:28:05.614903 20806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:05.615573 20806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:05.616770 20806 hybrid_clock.cc:648] HybridClock initialized: now 1786213685616749 us; error 41 us; skew 500 ppm
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213685, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:28:05.619904 20806 init.cc:377] Logged in from keytab as oryx/127.7.164.65@KRBTEST.COM (short username oryx)
I20260808 18:28:05.620945 20806 webserver.cc:460] Webserver started at http://127.7.164.65:40661/ using document root <none> and password file <none>
I20260808 18:28:05.621563 20806 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:05.621678 20806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:05.625705 20806 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:28:05.627928 20855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:05.629217 20806 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:28:05.629313 20806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "251828446911498ab52424524397a388"
format_stamp: "Formatted at 2026-08-08 18:28:04 on dist-test-slave-jztv"
server_key: "eee04f86c4a96706781d8c3f6a12b3d9"
server_key_iv: "fa2ba66ee4eec778bde208cd60725ba0"
server_key_version: "encryptionkey@0"
I20260808 18:28:05.629911 20806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:05.646600 20806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:05.649961 20806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:05.650192 20806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:05.650898 20806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:05.651814 20806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:05.651863 20806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:05.651904 20806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:05.651957 20806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:05.661224 20806 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:36073
I20260808 18:28:05.661247 21009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:36073 every 8 connection(s)
I20260808 18:28:05.662297 20806 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:28:05.670807 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 20806
I20260808 18:28:05.671351 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:36703
--local_ip_for_outbound_sockets=127.7.164.66
--tserver_master_addrs=127.7.164.126:33119
--webserver_port=40773
--webserver_interface=127.7.164.66
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:45609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.164.66
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.66.keytab with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:05.675928 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:05.664580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:36103)
0808 18:28:05.665064 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:05.665107 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:05.666371 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:05.667477 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:05.667483 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:05.667840 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:05.668309 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:05.668322 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.669585 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.669588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:05.670534 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:05.670544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.671668 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.671671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:05.671808 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:05.672437 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:05.672453 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:05.675168 (+ 2715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":132,"threads_started":1}
W20260808 18:28:05.676445 21011 heartbeater.cc:644] Failed to heartbeat to 127.7.164.126:33119 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.164.126:33119: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
W20260808 18:28:05.676445 21013 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:05.664431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:36103 (local address 127.7.164.126:33119)
0808 18:28:05.664897 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:05.664917 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:05.665223 (+ 306us) server_negotiation.cc:407] Connection header received
0808 18:28:05.666784 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:05.666796 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:05.667103 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:05.667395 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:05.668497 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.669463 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.670655 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.671390 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.671590 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:05.675481 (+ 3891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:36103: BlockingRecv error: recv got EOF from 127.7.164.65:36103 (error 108)
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":100,"threads_started":1}
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:05.683053 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:05.677058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:52527)
0808 18:28:05.677316 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:05.677328 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:05.677396 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:05.677681 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:05.677683 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:05.677817 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:05.677911 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:05.677915 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.679067 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.679070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:05.679614 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:05.679622 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.680269 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.680272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:05.680349 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:05.680877 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:05.680889 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:05.682943 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192}
W20260808 18:28:05.683302 21013 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:05.677190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:52527 (local address 127.7.164.126:33119)
0808 18:28:05.677351 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:05.677358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:05.677376 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:28:05.677490 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:05.677496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:05.677560 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:05.677660 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:05.678028 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.678957 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.679736 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.680140 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.680274 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:05.683147 (+ 2873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:52527: BlockingRecv error: recv got EOF from 127.7.164.65:52527 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:05.689244 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:05.683696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:56225)
0808 18:28:05.683912 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:05.683940 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:05.684004 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:05.684256 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:05.684258 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:05.684384 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:05.684456 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:05.684460 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.685521 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.685523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:05.686017 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:05.686024 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.686629 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.686651 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:05.686762 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:05.687283 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:05.687295 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:05.689149 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142}
W20260808 18:28:05.689401 21013 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:05.683819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:56225 (local address 127.7.164.126:33119)
0808 18:28:05.683968 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:05.683974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:05.683992 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:28:05.684075 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:05.684081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:05.684138 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:05.684233 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:05.684570 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.685401 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.686122 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.686500 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.686598 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:05.689291 (+ 2693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:56225: BlockingRecv error: recv got EOF from 127.7.164.65:56225 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 18:28:05.689636 21011 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:28:05.789978 21015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:05.790274 21015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:05.790338 21015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:05.794232 21015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:05.794277 21015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:05.794333 21015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:28:05.799404 21015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:45609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.66.keytab
--principal=oryx/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:36703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=40773
--tserver_master_addrs=127.7.164.126:33119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:05.800321 21015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:05.801239 21015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:05.810556 21026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:05.810562 21023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:05.810549 21022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:05.811010 21015 server_base.cc:1034] running on GCE node
I20260808 18:28:05.811414 21015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:05.812038 21015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:05.813216 21015 hybrid_clock.cc:648] HybridClock initialized: now 1786213685813203 us; error 34 us; skew 500 ppm
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213685, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:28:05.816020 21015 init.cc:377] Logged in from keytab as oryx/127.7.164.66@KRBTEST.COM (short username oryx)
I20260808 18:28:05.816915 21015 webserver.cc:460] Webserver started at http://127.7.164.66:40773/ using document root <none> and password file <none>
I20260808 18:28:05.817479 21015 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:05.817596 21015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:05.821518 21015 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:28:05.823585 21033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:05.824764 21015 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:28:05.824909 21015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "0fdf7d0694a2460bb0d9d0177ef70469"
format_stamp: "Formatted at 2026-08-08 18:28:04 on dist-test-slave-jztv"
server_key: "25738156a78b7f23cd1518d3761d72ad"
server_key_iv: "949324f1e46fd65382347aa77f592c48"
server_key_version: "encryptionkey@0"
I20260808 18:28:05.825440 21015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:05.848416 21015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:05.851722 21015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:05.851951 21015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:05.852586 21015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:05.853466 21015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:05.853544 21015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:05.853621 21015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:05.853665 21015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:05.864137 21015 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:36703
I20260808 18:28:05.864156 21202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:36703 every 8 connection(s)
I20260808 18:28:05.865383 21015 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:28:05.869731 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 21015
I20260808 18:28:05.870354 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:43677
--local_ip_for_outbound_sockets=127.7.164.67
--tserver_master_addrs=127.7.164.126:33119
--webserver_port=40267
--webserver_interface=127.7.164.67
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:45609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.164.67
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.67.keytab with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 18:28:05.877861 21013 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:05.867332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:50755 (local address 127.7.164.126:33119)
0808 18:28:05.867479 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:05.867483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:05.868287 (+ 804us) server_negotiation.cc:407] Connection header received
0808 18:28:05.869615 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:05.869618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:05.869685 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:05.869796 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:05.871572 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.872106 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.873271 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.873552 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.873674 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:05.877680 (+ 4006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:50755: BlockingRecv error: recv got EOF from 127.7.164.66:50755 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:28:05.878356 21207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:05.867667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:50755)
0808 18:28:05.868127 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:05.868169 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:05.869398 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:05.870010 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:05.870038 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:05.870646 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:05.871412 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:05.871429 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.872242 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.872245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:05.873134 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:05.873144 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.873750 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.873754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:05.873919 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:05.874638 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:05.874655 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:05.877462 (+ 2807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":131,"threads_started":1}
W20260808 18:28:05.878979 21203 heartbeater.cc:644] Failed to heartbeat to 127.7.164.126:33119 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.164.126:33119: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:05.884773 21207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:05.879489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:55591)
0808 18:28:05.879619 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:05.879635 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:05.879715 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:05.880031 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:05.880034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:05.880200 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:05.880287 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:05.880292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.881023 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.881027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:05.881574 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:05.881583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.882038 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.882041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:05.882129 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:05.882713 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:05.882725 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:05.884629 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 18:28:05.884881 21013 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:05.879688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:55591 (local address 127.7.164.126:33119)
0808 18:28:05.879842 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:05.879846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:05.879858 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:28:05.879892 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:05.879894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:05.879929 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:05.880014 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:05.880409 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.880913 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.881698 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.881917 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.882024 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:05.884749 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:55591: BlockingRecv error: recv got EOF from 127.7.164.66:55591 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:05 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:05.891582 21207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:05.885508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:47155)
0808 18:28:05.885737 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:05.885748 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:05.885817 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:05.886074 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:05.886076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:05.886198 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:05.886491 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:05.886497 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.887563 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.887566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:05.888069 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:05.888077 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.888622 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.888624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:05.888680 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:05.889205 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:05.889216 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:05.891463 (+ 2247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173}
W20260808 18:28:05.891757 21013 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:05.885623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:47155 (local address 127.7.164.126:33119)
0808 18:28:05.885750 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:05.885754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:05.885788 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:28:05.885920 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:05.885923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:05.885968 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:05.886045 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:05.886650 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.887447 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.888179 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:05.888516 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:05.888616 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:05.891638 (+ 3022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:47155: BlockingRecv error: recv got EOF from 127.7.164.66:47155 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 18:28:05.891830 21203 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:28:05.989424 21209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:05.989778 21209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:05.989868 21209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:05.994073 21209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:05.994189 21209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:05.994324 21209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:28:05.999904 21209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:45609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/krb5kdc/oryx_127.7.164.67.keytab
--principal=oryx/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:43677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=40267
--tserver_master_addrs=127.7.164.126:33119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:06.001008 21209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:06.002141 21209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:06.012148 21216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:06.012156 21219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:06.012208 21209 server_base.cc:1034] running on GCE node
W20260808 18:28:06.012148 21217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:06.012871 21209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:06.013667 21209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:06.014850 21209 hybrid_clock.cc:648] HybridClock initialized: now 1786213686014823 us; error 49 us; skew 500 ppm
Aug 08 18:28:06 dist-test-slave-jztv krb5kdc[19946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213686, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:28:06.017841 21209 init.cc:377] Logged in from keytab as oryx/127.7.164.67@KRBTEST.COM (short username oryx)
I20260808 18:28:06.018839 21209 webserver.cc:460] Webserver started at http://127.7.164.67:40267/ using document root <none> and password file <none>
I20260808 18:28:06.019394 21209 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:06.019479 21209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:06.023386 21209 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:28:06.025542 21231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:06.026966 21209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:28:06.027125 21209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "ee030f68bc924c77804f850639917a87"
format_stamp: "Formatted at 2026-08-08 18:28:05 on dist-test-slave-jztv"
server_key: "2806c42386a7a560c47e1f8fcb79c298"
server_key_iv: "c1707ceb3067ce7c1e29337fb330b09a"
server_key_version: "encryptionkey@0"
I20260808 18:28:06.027726 21209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:06.041177 21209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:06.044637 21209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:06.044862 21209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:06.045490 21209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:06.046444 21209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:06.046521 21209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:06.046595 21209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:06.046648 21209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:06.056744 21209 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:43677
I20260808 18:28:06.056766 21388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:43677 every 8 connection(s)
I20260808 18:28:06.057829 21209 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:28:06.058308 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 21209
Aug 08 18:28:06 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:06.069761 21013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:06.059177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.7.164.126:33119)
0808 18:28:06.059337 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:06.059341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:06.059837 (+ 496us) server_negotiation.cc:407] Connection header received
0808 18:28:06.059937 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:06.059944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:06.060013 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:06.060104 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:06.061130 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.061830 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.063117 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.063415 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.063496 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:06.065251 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:06.065258 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:06.065275 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:28:06.065309 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:06.067315 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:06.067882 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:06.067887 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:06.067892 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:06.068029 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:06.068417 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:06.068421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:06.068425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:06.068784 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:06.069014 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:06.069369 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 18:28:06.069632 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 18:28:06 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 18:28:06.071213 21394 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:06.060534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:43459 (local address 127.7.164.126:33119)
0808 18:28:06.060796 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:06.060800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:06.061348 (+ 548us) server_negotiation.cc:407] Connection header received
0808 18:28:06.063028 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:06.063030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:06.063112 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:06.063206 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:06.064854 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.065567 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.067028 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.067417 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.067511 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:06.071117 (+ 3606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:43459: BlockingRecv error: recv got EOF from 127.7.164.67:43459 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":76,"threads_started":1}
I20260808 18:28:06.071967 21395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:06.060671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:43459)
0808 18:28:06.061208 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:06.061251 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:06.062791 (+ 1540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:06.063375 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:06.063382 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:06.063926 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:06.064673 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:06.064700 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.065729 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.065734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:06.066895 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:06.066909 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.067577 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.067582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:06.067804 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:06.068407 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:06.068423 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:06.071016 (+ 2593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":126,"threads_started":1}
W20260808 18:28:06.072494 21391 heartbeater.cc:644] Failed to heartbeat to 127.7.164.126:33119 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.164.126:33119: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Aug 08 18:28:06 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:06 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:06.078152 21395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:06.072967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:56381)
0808 18:28:06.073149 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:06.073165 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:06.073261 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:06.073521 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:06.073523 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:06.073651 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:06.073740 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:06.073745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.074509 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.074512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:06.075034 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:06.075041 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.075503 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.075506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:06.075571 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:06.076066 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:06.076078 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:06.077968 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111}
W20260808 18:28:06.078770 21394 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:06.073050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:56381 (local address 127.7.164.126:33119)
0808 18:28:06.073170 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:06.073174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:06.073221 (+ 47us) server_negotiation.cc:407] Connection header received
0808 18:28:06.073366 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:06.073369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:06.073420 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:06.073488 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:06.073835 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.074416 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.075152 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.075382 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.075490 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:06.078629 (+ 3139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:56381: BlockingRecv error: recv got EOF from 127.7.164.67:56381 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 18:28:06 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:06 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:06.083187 21395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:06.078696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:47741)
0808 18:28:06.078962 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:06.078977 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:06.079059 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:06.079252 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:06.079254 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:06.079386 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:06.079454 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:06.079458 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.080101 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.080103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:06.080603 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:06.080611 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.080988 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.080991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:06.081047 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:06.081434 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:06.081445 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:06.083100 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200}
W20260808 18:28:06.083390 21391 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:28:06.083424 21394 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:06.078821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:47741 (local address 127.7.164.126:33119)
0808 18:28:06.078991 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:06.078995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:06.079015 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:28:06.079139 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:06.079142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:06.079179 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:06.079237 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:06.079546 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.080012 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.080699 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.080896 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.080968 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:06.083296 (+ 2328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:47741: BlockingRecv error: recv got EOF from 127.7.164.67:47741 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 08 18:28:06 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:06 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:06.696333 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:06.690410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:50893)
0808 18:28:06.690701 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:06.690716 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:06.690816 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:06.691232 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:06.691235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:06.691420 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:06.691555 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:06.691561 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.692321 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.692323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:06.692897 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:06.692905 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.693332 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.693334 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:06.693403 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:06.694013 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:06.694025 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:06.696214 (+ 2189us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":78,"threads_started":1}
W20260808 18:28:06.696512 21398 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:06.690426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:50893 (local address 127.7.164.126:33119)
0808 18:28:06.690753 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:06.690770 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:06.690787 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:28:06.690951 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:06.690954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:06.691006 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:06.691139 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:06.691678 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.692225 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.693007 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.693228 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.693308 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:06.696370 (+ 3062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:50893: BlockingRecv error: recv got EOF from 127.7.164.65:50893 (error 108)
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":119,"threads_started":1}
Aug 08 18:28:06 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:06 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:06.898730 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:06.892563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:50957)
0808 18:28:06.892849 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:06.892863 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:06.892968 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:06.893246 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:06.893248 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:06.893450 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:06.893552 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:06.893558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.894777 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.894779 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:06.895437 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:06.895448 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.895906 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.895909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:06.895974 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:06.896617 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:06.896628 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:06.898605 (+ 1977us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":99,"threads_started":1}
W20260808 18:28:06.898900 21398 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:06.892677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:50957 (local address 127.7.164.126:33119)
0808 18:28:06.892844 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:06.892849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:06.892942 (+ 93us) server_negotiation.cc:407] Connection header received
0808 18:28:06.893081 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:06.893084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:06.893136 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:06.893210 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:06.893719 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.894650 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.895563 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:06.895801 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:06.895895 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:06.898762 (+ 2867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:50957: BlockingRecv error: recv got EOF from 127.7.164.66:50957 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 18:28:07 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:07 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:07.089576 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:07.084068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:51663)
0808 18:28:07.084373 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:07.084396 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:07.084501 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:07.084762 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:07.084764 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:07.084964 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:07.085066 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:07.085072 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:07.085815 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:07.085818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:07.086484 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:07.086494 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:07.086920 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:07.086922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:07.086986 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:07.087580 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:07.087593 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:07.089444 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":128,"threads_started":1}
W20260808 18:28:07.089778 21398 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:07.084174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:51663 (local address 127.7.164.126:33119)
0808 18:28:07.084324 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:07.084328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:07.084471 (+ 143us) server_negotiation.cc:407] Connection header received
0808 18:28:07.084596 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:07.084599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:07.084646 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:07.084718 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:07.085171 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:07.085697 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:07.086597 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:07.086817 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:07.086899 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:07.089605 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:51663: BlockingRecv error: recv got EOF from 127.7.164.67:51663 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 18:28:07 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:07 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:07.703161 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:07.697256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:44017)
0808 18:28:07.697666 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:07.697680 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:07.697776 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:07.698079 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:07.698082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:07.698298 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:07.698403 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:07.698408 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:07.699226 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:07.699230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:07.699813 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:07.699822 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:07.700344 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:07.700347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:07.700445 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:07.700955 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:07.700966 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:07.703024 (+ 2058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":177,"threads_started":1}
W20260808 18:28:07.703451 21404 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:07.697416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:44017 (local address 127.7.164.126:33119)
0808 18:28:07.697744 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:07.697747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:07.697762 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:28:07.697873 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:07.697876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:07.697923 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:07.698003 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:07.698528 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:07.699090 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:07.699945 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:07.700179 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:07.700265 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:07.703182 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:44017: BlockingRecv error: recv got EOF from 127.7.164.65:44017 (error 108)
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":128,"threads_started":1}
Aug 08 18:28:07 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:07 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:07.904732 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:07.899584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:37839)
0808 18:28:07.899842 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:07.899856 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:07.899950 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:07.900179 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:07.900182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:07.900366 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:07.900465 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:07.900471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:07.901192 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:07.901194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:07.901731 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:07.901739 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:07.902193 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:07.902196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:07.902278 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:07.902751 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:07.902762 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:07.904594 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":70,"threads_started":1}
W20260808 18:28:07.904920 21404 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:07.899728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:37839 (local address 127.7.164.126:33119)
0808 18:28:07.899885 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:07.899888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:07.899903 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:28:07.900034 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:07.900036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:07.900080 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:07.900146 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:07.900561 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:07.901100 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:07.901859 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:07.902084 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:07.902175 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:07.904753 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:37839: BlockingRecv error: recv got EOF from 127.7.164.66:37839 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 18:28:08 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:08 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:08.096068 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:08.090424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:37555)
0808 18:28:08.090811 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:08.090824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:08.090922 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:08.091176 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:08.091178 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:08.091362 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:08.091476 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:08.091481 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:08.092208 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:08.092210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:08.092803 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:08.092812 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:08.093239 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:08.093241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:08.093301 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:08.093789 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:08.093800 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:08.095867 (+ 2067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":122,"threads_started":1}
W20260808 18:28:08.096108 21404 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:08.090644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:37555 (local address 127.7.164.126:33119)
0808 18:28:08.090794 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:08.090798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:08.090891 (+ 93us) server_negotiation.cc:407] Connection header received
0808 18:28:08.091003 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:08.091005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:08.091067 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:08.091136 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:08.091581 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:08.092116 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:08.092913 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:08.093136 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:08.093219 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:08.096004 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:37555: BlockingRecv error: recv got EOF from 127.7.164.67:37555 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 18:28:08 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:08 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:08.709522 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:08.704157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:42841)
0808 18:28:08.704433 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:08.704447 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:08.704537 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:08.704779 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:08.704780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:08.704953 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:08.705049 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:08.705055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:08.705821 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:08.705824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:08.706519 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:08.706529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:08.706943 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:08.706945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:08.707002 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:08.707552 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:08.707565 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:08.709395 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":103,"threads_started":1}
W20260808 18:28:08.709805 21409 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:08.704216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:42841 (local address 127.7.164.126:33119)
0808 18:28:08.704504 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:08.704508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:08.704523 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:28:08.704635 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:08.704638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:08.704682 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:08.704769 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:08.705160 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:08.705719 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:08.706633 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:08.706846 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:08.706925 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:08.709668 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:42841: BlockingRecv error: recv got EOF from 127.7.164.65:42841 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":110,"threads_started":1}
Aug 08 18:28:08 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:08 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:08.912051 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:08.905656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:36233)
0808 18:28:08.906000 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:08.906013 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:08.906135 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:08.906498 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:08.906500 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:08.906699 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:08.906803 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:08.906809 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:08.907812 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:08.907819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:08.908749 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:08.908760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:08.909230 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:08.909233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:08.909314 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:08.909882 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:08.909896 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:08.911930 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":127,"threads_started":1}
W20260808 18:28:08.912199 21409 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:08.905747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:36233 (local address 127.7.164.126:33119)
0808 18:28:08.905922 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:08.905929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:08.906138 (+ 209us) server_negotiation.cc:407] Connection header received
0808 18:28:08.906301 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:08.906304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:08.906356 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:08.906429 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:08.906981 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:08.907657 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:08.908870 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:08.909109 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:08.909228 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:08.912077 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:36233: BlockingRecv error: recv got EOF from 127.7.164.66:36233 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 18:28:09 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:09 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:09.102670 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:09.096944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:47621)
0808 18:28:09.097290 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:09.097306 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:09.097405 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:09.097643 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:09.097645 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:09.097831 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:09.097927 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:09.097933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:09.098891 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:09.098894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:09.099466 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:09.099480 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:09.099980 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:09.099984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:09.100073 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:09.100597 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:09.100609 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:09.102550 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":113,"threads_started":1}
W20260808 18:28:09.102824 21409 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:09.097078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:47621 (local address 127.7.164.126:33119)
0808 18:28:09.097215 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:09.097219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:09.097379 (+ 160us) server_negotiation.cc:407] Connection header received
0808 18:28:09.097493 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:09.097495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:09.097545 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:09.097609 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:09.098049 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:09.098769 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:09.099614 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:09.099857 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:09.099982 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:09.102702 (+ 2720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:47621: BlockingRecv error: recv got EOF from 127.7.164.67:47621 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 18:28:09 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:09 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:09.722357 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:09.710473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:59701)
0808 18:28:09.710790 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:09.710805 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:09.710910 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:09.711185 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:09.711187 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:09.711370 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:09.711474 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:09.711479 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:09.712373 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:09.712376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:09.713079 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:09.713090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:09.713588 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:09.713591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:09.713716 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:09.720069 (+ 6353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:09.720082 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:09.722160 (+ 2078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":86,"threads_started":1}
W20260808 18:28:09.722541 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:09.710483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:59701 (local address 127.7.164.126:33119)
0808 18:28:09.710826 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:09.710829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:09.710861 (+ 32us) server_negotiation.cc:407] Connection header received
0808 18:28:09.711012 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:09.711014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:09.711061 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:09.711143 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:09.711601 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:09.712211 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:09.713219 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:09.713453 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:09.713545 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:09.722388 (+ 8843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:59701: BlockingRecv error: recv got EOF from 127.7.164.65:59701 (error 108)
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":144,"threads_started":1}
Aug 08 18:28:09 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:09 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:09.918808 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:09.913011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:59183)
0808 18:28:09.913293 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:09.913307 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:09.913414 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:09.913687 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:09.913690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:09.913879 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:09.914004 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:09.914010 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:09.914888 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:09.914891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:09.915453 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:09.915462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:09.915883 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:09.915885 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:09.915947 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:09.916582 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:09.916594 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:09.918643 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
W20260808 18:28:09.918958 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:09.913094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:59183 (local address 127.7.164.126:33119)
0808 18:28:09.913253 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:09.913257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:09.913380 (+ 123us) server_negotiation.cc:407] Connection header received
0808 18:28:09.913490 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:09.913492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:09.913554 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:09.913653 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:09.914143 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:09.914774 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:09.915563 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:09.915780 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:09.915859 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:09.918807 (+ 2948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:59183: BlockingRecv error: recv got EOF from 127.7.164.66:59183 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 18:28:10 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:10 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:10.109272 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:10.103641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:47509)
0808 18:28:10.103921 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:10.103934 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:10.104037 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:10.104289 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:10.104291 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:10.104474 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:10.104573 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:10.104579 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:10.105350 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:10.105352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:10.105935 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:10.105944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:10.106472 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:10.106475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:10.106558 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:10.107055 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:10.107067 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:10.109092 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":97,"threads_started":1}
W20260808 18:28:10.109339 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:10.103701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:47509 (local address 127.7.164.126:33119)
0808 18:28:10.103863 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:10.103867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:10.104007 (+ 140us) server_negotiation.cc:407] Connection header received
0808 18:28:10.104119 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:10.104122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:10.104171 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:10.104245 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:10.104704 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:10.105237 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:10.106072 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:10.106328 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:10.106440 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:10.109217 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:47509: BlockingRecv error: recv got EOF from 127.7.164.67:47509 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 18:28:10 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:10 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:10.729486 21418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:10.723228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:44849)
0808 18:28:10.723547 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:10.723563 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:10.723670 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:10.723945 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:10.723947 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:10.724154 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:10.724260 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:10.724266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:10.725038 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:10.725040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:10.725685 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:10.725694 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:10.726379 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:10.726388 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:10.726471 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:10.726997 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:10.727010 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:10.729303 (+ 2293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":94,"threads_started":1}
W20260808 18:28:10.729700 21419 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:10.723404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:44849 (local address 127.7.164.126:33119)
0808 18:28:10.723632 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:10.723635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:10.723649 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:28:10.723766 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:10.723768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:10.723811 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:10.723900 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:10.724380 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:10.724936 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:10.725826 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:10.726068 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:10.726158 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:10.729511 (+ 3353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:44849: BlockingRecv error: recv got EOF from 127.7.164.65:44849 (error 108)
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":71,"threads_started":1}
Aug 08 18:28:10 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:10 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:10.925040 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:10.919586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:49137)
0808 18:28:10.919864 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:10.919877 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:10.919964 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:10.920201 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:10.920203 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:10.920391 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:10.920505 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:10.920511 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:10.921240 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:10.921242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:10.921796 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:10.921803 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:10.922315 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:10.922318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:10.922378 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:10.922906 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:10.922917 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:10.924864 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":102,"threads_started":1}
W20260808 18:28:10.925125 21419 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:10.919748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:49137 (local address 127.7.164.126:33119)
0808 18:28:10.919900 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:10.919903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:10.919920 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:28:10.920046 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:10.920048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:10.920092 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:10.920162 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:10.920607 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:10.921146 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:10.921914 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:10.922154 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:10.922269 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:10.925008 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:49137: BlockingRecv error: recv got EOF from 127.7.164.66:49137 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 18:28:11 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:11 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:11.116333 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:11.110140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:42177)
0808 18:28:11.110469 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:11.110483 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:11.110589 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:11.110869 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:11.110871 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:11.111105 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:11.111215 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:11.111222 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:11.111982 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:11.111985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:11.112618 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:11.112628 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:11.113139 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:11.113142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:11.113264 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:11.113889 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:11.113908 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:11.116154 (+ 2246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":145,"threads_started":1}
W20260808 18:28:11.116381 21419 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:11.110331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:42177 (local address 127.7.164.126:33119)
0808 18:28:11.110483 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:11.110487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:11.110553 (+ 66us) server_negotiation.cc:407] Connection header received
0808 18:28:11.110671 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:11.110673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:11.110719 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:11.110811 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:11.111322 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:11.111844 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:11.112725 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:11.112975 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:11.113142 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:11.116278 (+ 3136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:42177: BlockingRecv error: recv got EOF from 127.7.164.67:42177 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 18:28:11 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:11 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:11.736820 21423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:11.730408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:48925)
0808 18:28:11.730715 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:11.730731 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:11.730832 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:11.731266 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:11.731268 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:11.731556 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:11.731668 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:11.731675 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:11.732777 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:11.732781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:11.733453 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:11.733464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:11.733996 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:11.733999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:11.734068 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:11.734754 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:11.734775 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:11.736697 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":105,"threads_started":1}
W20260808 18:28:11.737040 21428 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:11.730474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:48925 (local address 127.7.164.126:33119)
0808 18:28:11.730766 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:11.730770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:11.730786 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:28:11.730994 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:11.731000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:11.731089 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:11.731204 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:11.731807 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:11.732600 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:11.733590 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:11.733829 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:11.734003 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:11.736876 (+ 2873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:48925: BlockingRecv error: recv got EOF from 127.7.164.65:48925 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":100,"threads_started":1}
Aug 08 18:28:11 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:11 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:11.931901 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:11.926010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:37469)
0808 18:28:11.926375 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:11.926390 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:11.926500 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:11.926765 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:11.926768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:11.926979 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:11.927083 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:11.927088 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:11.927913 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:11.927916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:11.928510 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:11.928519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:11.929048 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:11.929052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:11.929163 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:11.929746 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:11.929758 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:11.931769 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":116,"threads_started":1}
W20260808 18:28:11.932040 21428 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:11.926066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:37469 (local address 127.7.164.126:33119)
0808 18:28:11.926274 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:11.926279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:11.926471 (+ 192us) server_negotiation.cc:407] Connection header received
0808 18:28:11.926573 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:11.926575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:11.926626 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:11.926702 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:11.927198 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:11.927789 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:11.928646 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:11.928879 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:11.929049 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:11.931916 (+ 2867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:37469: BlockingRecv error: recv got EOF from 127.7.164.66:37469 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 18:28:12 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:12 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:12.122869 21432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:12.117366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:55921)
0808 18:28:12.117668 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:12.117681 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:12.117788 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:12.118021 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:12.118024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:12.118283 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:12.118412 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:12.118419 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:12.119141 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:12.119144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:12.119728 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:12.119736 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:12.120146 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:12.120148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:12.120209 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:12.120762 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:12.120775 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:12.122720 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":98,"threads_started":1}
W20260808 18:28:12.123071 21428 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:12.117457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:55921 (local address 127.7.164.126:33119)
0808 18:28:12.117598 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:12.117605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:12.117754 (+ 149us) server_negotiation.cc:407] Connection header received
0808 18:28:12.117863 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:12.117865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:12.117912 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:12.117979 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:12.118515 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:12.119033 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:12.119842 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:12.120051 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:12.120127 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:12.122901 (+ 2774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:55921: BlockingRecv error: recv got EOF from 127.7.164.67:55921 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 18:28:12 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:12 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:12.743662 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:12.737682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:57645)
0808 18:28:12.738042 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:12.738058 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:12.738207 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:12.738516 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:12.738520 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:12.738739 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:12.738832 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:12.738837 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:12.739658 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:12.739664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:12.740236 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:12.740244 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:12.740720 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:12.740723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:12.740810 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:12.741385 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:12.741398 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:12.743481 (+ 2083us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":99,"threads_started":1}
W20260808 18:28:12.743726 21434 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:12.737748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:57645 (local address 127.7.164.126:33119)
0808 18:28:12.738041 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:12.738045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:12.738145 (+ 100us) server_negotiation.cc:407] Connection header received
0808 18:28:12.738326 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:12.738330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:12.738392 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:12.738490 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:12.738963 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:12.739546 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:12.740367 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:12.740603 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:12.740714 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:12.743603 (+ 2889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:57645: BlockingRecv error: recv got EOF from 127.7.164.65:57645 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":109,"threads_started":1}
Aug 08 18:28:12 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:12 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:12.938719 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:12.932794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:40709)
0808 18:28:12.933170 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:12.933187 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:12.933317 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:12.933546 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:12.933548 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:12.933742 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:12.933845 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:12.933850 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:12.934716 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:12.934719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:12.935290 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:12.935299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:12.935724 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:12.935728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:12.935795 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:12.936318 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:12.936330 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:12.938601 (+ 2271us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":110,"threads_started":1}
W20260808 18:28:12.938860 21434 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:12.932859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:40709 (local address 127.7.164.126:33119)
0808 18:28:12.933005 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:12.933009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:12.933287 (+ 278us) server_negotiation.cc:407] Connection header received
0808 18:28:12.933385 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:12.933387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:12.933435 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:12.933505 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:12.933960 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:12.934605 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:12.935395 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:12.935614 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:12.935689 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:12.938742 (+ 3053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:40709: BlockingRecv error: recv got EOF from 127.7.164.66:40709 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 18:28:13 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:13 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:13.130391 21437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:13.124050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:45963)
0808 18:28:13.124406 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:13.124421 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:13.124564 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:13.125020 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:13.125030 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:13.125304 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:13.125441 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:13.125450 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:13.126396 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:13.126400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:13.126957 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:13.126966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:13.127440 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:13.127444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:13.127515 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:13.128056 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:13.128069 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:13.130267 (+ 2198us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":113,"threads_started":1}
W20260808 18:28:13.130553 21434 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:13.124123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:45963 (local address 127.7.164.126:33119)
0808 18:28:13.124290 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:13.124294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:13.124506 (+ 212us) server_negotiation.cc:407] Connection header received
0808 18:28:13.124681 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:13.124684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:13.124815 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:13.124907 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:13.125630 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:13.126199 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:13.127079 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:13.127305 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:13.127389 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:13.130421 (+ 3032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:45963: BlockingRecv error: recv got EOF from 127.7.164.67:45963 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 18:28:13 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:13 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:13.750666 21438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:13.744512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:37375)
0808 18:28:13.744859 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:13.744873 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:13.744965 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:13.745257 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:13.745260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:13.745568 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:13.745679 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:13.745686 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:13.746551 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:13.746554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:13.747122 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:13.747130 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:13.747632 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:13.747636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:13.747708 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:13.748304 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:13.748317 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:13.750484 (+ 2167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":131,"threads_started":1}
W20260808 18:28:13.750747 21440 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:13.744695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:37375 (local address 127.7.164.126:33119)
0808 18:28:13.744915 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:13.744918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:13.744932 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:28:13.745057 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:13.745059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:13.745104 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:13.745179 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:13.745825 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:13.746415 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:13.747242 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:13.747486 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:13.747567 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:13.750622 (+ 3055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:37375: BlockingRecv error: recv got EOF from 127.7.164.65:37375 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":63,"threads_started":1}
Aug 08 18:28:13 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:13 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:13.945592 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:13.939556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:35787)
0808 18:28:13.939859 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:13.939873 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:13.939975 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:13.940276 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:13.940278 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:13.940479 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:13.940579 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:13.940584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:13.941467 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:13.941470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:13.942017 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:13.942024 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:13.942554 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:13.942558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:13.942647 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:13.943279 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:13.943296 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:13.945413 (+ 2117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":106,"threads_started":1}
W20260808 18:28:13.945641 21440 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:13.939637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:35787 (local address 127.7.164.126:33119)
0808 18:28:13.939818 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:13.939822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:13.939948 (+ 126us) server_negotiation.cc:407] Connection header received
0808 18:28:13.940063 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:13.940065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:13.940112 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:13.940192 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:13.940684 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:13.941347 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:13.942118 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:13.942428 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:13.942547 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:13.945540 (+ 2993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:35787: BlockingRecv error: recv got EOF from 127.7.164.66:35787 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 18:28:14 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:14 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:14.137032 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:14.131206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:56577)
0808 18:28:14.131564 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:14.131577 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:14.131674 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:14.131903 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:14.131906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:14.132096 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:14.132191 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:14.132197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:14.133088 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:14.133091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:14.133731 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:14.133741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:14.134267 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:14.134271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:14.134344 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:14.134907 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:14.134921 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:14.136856 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":127,"threads_started":1}
W20260808 18:28:14.137104 21440 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:14.131282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:56577 (local address 127.7.164.126:33119)
0808 18:28:14.131458 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:14.131464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:14.131649 (+ 185us) server_negotiation.cc:407] Connection header received
0808 18:28:14.131753 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:14.131756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:14.131803 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:14.131862 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:14.132304 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:14.132985 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:14.133890 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:14.134117 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:14.134198 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:14.136993 (+ 2795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:56577: BlockingRecv error: recv got EOF from 127.7.164.67:56577 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 08 18:28:14 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:14 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:14.760766 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:14.751547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:41937)
0808 18:28:14.751849 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:14.751862 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:14.751961 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:14.752204 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:14.752207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:14.752388 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:14.752509 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:14.752515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:14.753326 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:14.753329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:14.753906 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:14.753915 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:14.754388 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:14.754391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:14.754480 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:14.758261 (+ 3781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:14.758274 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:14.760646 (+ 2372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":121,"threads_started":1}
W20260808 18:28:14.761000 21446 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:14.751614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:41937 (local address 127.7.164.126:33119)
0808 18:28:14.751868 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:14.751871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:14.751934 (+ 63us) server_negotiation.cc:407] Connection header received
0808 18:28:14.752038 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:14.752040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:14.752086 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:14.752169 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:14.752651 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:14.753195 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:14.754029 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:14.754289 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:14.754378 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:14.760848 (+ 6470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:41937: BlockingRecv error: recv got EOF from 127.7.164.65:41937 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":93,"threads_started":1}
Aug 08 18:28:14 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:14 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:14.952214 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:14.946472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:45257)
0808 18:28:14.946736 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:14.946750 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:14.946850 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:14.947098 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:14.947100 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:14.947283 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:14.947389 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:14.947395 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:14.948252 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:14.948260 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:14.948822 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:14.948832 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:14.949323 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:14.949327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:14.949409 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:14.950014 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:14.950027 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:14.952069 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":94,"threads_started":1}
W20260808 18:28:14.952299 21446 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:14.946558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:45257 (local address 127.7.164.126:33119)
0808 18:28:14.946702 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:14.946706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:14.946817 (+ 111us) server_negotiation.cc:407] Connection header received
0808 18:28:14.946936 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:14.946938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:14.946992 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:14.947059 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:14.947519 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:14.948103 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:14.948953 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:14.949173 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:14.949254 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:14.952204 (+ 2950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:45257: BlockingRecv error: recv got EOF from 127.7.164.66:45257 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 18:28:15 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:15 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:15.143451 21451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:15.137837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:39247)
0808 18:28:15.138230 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:15.138243 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:15.138350 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:15.138600 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:15.138602 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:15.138792 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:15.138891 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:15.138896 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:15.139651 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:15.139654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:15.140222 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:15.140230 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:15.140677 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:15.140680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:15.140746 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:15.141287 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:15.141299 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:15.143283 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":202,"threads_started":1}
W20260808 18:28:15.143527 21446 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:15.137983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:39247 (local address 127.7.164.126:33119)
0808 18:28:15.138129 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:15.138132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:15.138323 (+ 191us) server_negotiation.cc:407] Connection header received
0808 18:28:15.138434 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:15.138437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:15.138485 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:15.138559 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:15.139006 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:15.139542 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:15.140337 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:15.140566 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:15.140648 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:15.143428 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:39247: BlockingRecv error: recv got EOF from 127.7.164.67:39247 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 18:28:15 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:15 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:15.767364 21452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:15.761680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:42399)
0808 18:28:15.762011 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:15.762026 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:15.762114 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:15.762399 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:15.762401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:15.762589 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:15.762691 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:15.762696 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:15.763438 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:15.763440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:15.764097 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:15.764129 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:15.764728 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:15.764733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:15.764824 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:15.765418 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:15.765433 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:15.767247 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":78,"threads_started":1}
W20260808 18:28:15.767571 21453 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:15.761754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:42399 (local address 127.7.164.126:33119)
0808 18:28:15.762085 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:15.762089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:15.762103 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:28:15.762206 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:15.762230 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:15.762276 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:15.762358 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:15.762798 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:15.763339 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:15.764244 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:15.764595 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:15.764703 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:15.767397 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:42399: BlockingRecv error: recv got EOF from 127.7.164.65:42399 (error 108)
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":147,"threads_started":1}
Aug 08 18:28:15 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:15 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:15.958977 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:15.953104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:50573)
0808 18:28:15.953469 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:15.953485 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:15.953639 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:15.953941 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:15.953943 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:15.954180 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:15.954313 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:15.954319 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:15.955118 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:15.955123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:15.955692 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:15.955700 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:15.956186 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:15.956190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:15.956278 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:15.956863 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:15.956875 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:15.958857 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":91,"threads_started":1}
W20260808 18:28:15.959201 21453 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:15.953234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:50573 (local address 127.7.164.126:33119)
0808 18:28:15.953439 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:15.953444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:15.953587 (+ 143us) server_negotiation.cc:407] Connection header received
0808 18:28:15.953761 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:15.953764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:15.953818 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:15.953910 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:15.954453 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:15.955001 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:15.955827 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:15.956062 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:15.956178 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:15.959074 (+ 2896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:50573: BlockingRecv error: recv got EOF from 127.7.164.66:50573 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 08 18:28:16 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:16 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:16.150080 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:16.144313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:45453)
0808 18:28:16.144620 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:16.144634 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:16.144730 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:16.144956 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:16.144958 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:16.145142 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:16.145237 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:16.145242 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:16.146018 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:16.146021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:16.146604 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:16.146612 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:16.147105 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:16.147108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:16.147173 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:16.147796 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:16.147808 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:16.149904 (+ 2096us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":121,"threads_started":1}
W20260808 18:28:16.150146 21453 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:16.144469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:45453 (local address 127.7.164.126:33119)
0808 18:28:16.144647 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:16.144650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:16.144707 (+ 57us) server_negotiation.cc:407] Connection header received
0808 18:28:16.144812 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:16.144814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:16.144864 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:16.144938 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:16.145355 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:16.145916 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:16.146749 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:16.146993 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:16.147086 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:16.150037 (+ 2951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:45453: BlockingRecv error: recv got EOF from 127.7.164.67:45453 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 08 18:28:16 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:16 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:16.773988 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:16.768293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:57205)
0808 18:28:16.768620 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:16.768633 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:16.768731 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:16.768986 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:16.768988 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:16.769170 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:16.769307 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:16.769314 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:16.770344 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:16.770348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:16.770985 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:16.770996 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:16.771440 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:16.771444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:16.771513 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:16.771989 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:16.772000 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:16.773841 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":88,"threads_started":1}
W20260808 18:28:16.774122 21458 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:16.768448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:57205 (local address 127.7.164.126:33119)
0808 18:28:16.768684 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:16.768687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:16.768701 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:28:16.768819 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:16.768821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:16.768865 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:16.768951 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:16.769425 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:16.770195 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:16.771090 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:16.771310 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:16.771391 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:16.773988 (+ 2597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:57205: BlockingRecv error: recv got EOF from 127.7.164.65:57205 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":61,"threads_started":1}
Aug 08 18:28:16 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:16 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:16.965660 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:16.959757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:46215)
0808 18:28:16.960190 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:16.960211 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:16.960360 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:16.960649 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:16.960652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:16.960873 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:16.960994 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:16.961001 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:16.961799 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:16.961804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:16.962395 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:16.962404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:16.962873 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:16.962877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:16.962973 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:16.963519 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:16.963531 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:16.965537 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":94,"threads_started":1}
W20260808 18:28:16.965885 21458 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:16.959885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:46215 (local address 127.7.164.126:33119)
0808 18:28:16.960155 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:16.960160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:16.960298 (+ 138us) server_negotiation.cc:407] Connection header received
0808 18:28:16.960483 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:16.960491 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:16.960536 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:16.960621 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:16.961132 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:16.961683 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:16.962528 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:16.962755 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:16.962866 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:16.965724 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:46215: BlockingRecv error: recv got EOF from 127.7.164.66:46215 (error 108)
Metrics: {"server-negotiator.queue_time_us":166}
Aug 08 18:28:17 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:17 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:17.157060 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:17.150974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:36681)
0808 18:28:17.151374 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:17.151387 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:17.151528 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:17.151817 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:17.151820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:17.152043 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:17.152162 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:17.152169 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:17.152971 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:17.152974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:17.153526 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:17.153534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:17.154047 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:17.154051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:17.154158 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:17.154790 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:17.154802 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:17.156829 (+ 2027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":89,"threads_started":1}
W20260808 18:28:17.157092 21458 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:17.151031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:36681 (local address 127.7.164.126:33119)
0808 18:28:17.151307 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:17.151312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:17.151469 (+ 157us) server_negotiation.cc:407] Connection header received
0808 18:28:17.151635 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:17.151638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:17.151696 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:17.151787 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:17.152290 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:17.152837 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:17.153651 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:17.153917 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:17.154030 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:17.156984 (+ 2954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:36681: BlockingRecv error: recv got EOF from 127.7.164.67:36681 (error 108)
Metrics: {"server-negotiator.queue_time_us":178}
Aug 08 18:28:17 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:17 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:17.780398 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:17.774816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:48271)
0808 18:28:17.775124 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:17.775149 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:17.775247 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:17.775519 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:17.775522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:17.775710 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:17.775814 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:17.775819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:17.776651 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:17.776655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:17.777211 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:17.777220 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:17.777673 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:17.777675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:17.777739 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:17.778298 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:17.778320 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:17.780281 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":90,"threads_started":1}
W20260808 18:28:17.780614 21462 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:17.774979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:48271 (local address 127.7.164.126:33119)
0808 18:28:17.775240 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:17.775244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:17.775258 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:28:17.775328 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:17.775330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:17.775373 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:17.775460 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:17.775940 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:17.776501 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:17.777326 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:17.777561 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:17.777640 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:17.780421 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:48271: BlockingRecv error: recv got EOF from 127.7.164.65:48271 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":79,"threads_started":1}
Aug 08 18:28:17 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:17 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:17.972661 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:17.966704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:48967)
0808 18:28:17.967014 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:17.967030 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:17.967145 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:17.967478 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:17.967482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:17.967737 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:17.967900 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:17.967909 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:17.968985 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:17.968990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:17.969604 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:17.969614 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:17.970049 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:17.970052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:17.970119 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:17.970634 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:17.970647 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:17.972543 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":123,"threads_started":1}
W20260808 18:28:17.972802 21462 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:17.966802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:48967 (local address 127.7.164.126:33119)
0808 18:28:17.966997 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:17.967006 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:17.967114 (+ 108us) server_negotiation.cc:407] Connection header received
0808 18:28:17.967229 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:17.967232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:17.967276 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:17.967361 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:17.968067 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:17.968840 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:17.969722 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:17.969946 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:17.970032 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:17.972680 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:48967: BlockingRecv error: recv got EOF from 127.7.164.66:48967 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 08 18:28:18 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:18 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:18.163483 21464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:18.157972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:45561)
0808 18:28:18.158262 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:18.158276 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:18.158402 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:18.158681 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:18.158684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:18.158889 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:18.158995 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:18.159001 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:18.159809 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:18.159813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:18.160394 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:18.160402 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:18.160821 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:18.160824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:18.160892 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:18.161425 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:18.161438 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:18.163351 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":103,"threads_started":1}
W20260808 18:28:18.163618 21462 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:18.158050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:45561 (local address 127.7.164.126:33119)
0808 18:28:18.158183 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:18.158186 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:18.158347 (+ 161us) server_negotiation.cc:407] Connection header received
0808 18:28:18.158486 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:18.158488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:18.158535 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:18.158603 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:18.159128 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:18.159689 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:18.160499 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:18.160717 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:18.160788 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:18.163495 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:45561: BlockingRecv error: recv got EOF from 127.7.164.67:45561 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 18:28:18 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:18 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:18.786761 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:18.781301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:46201)
0808 18:28:18.781673 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:18.781687 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:18.781787 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:18.782032 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:18.782034 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:18.782246 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:18.782345 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:18.782351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:18.783108 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:18.783111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:18.783685 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:18.783693 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:18.784120 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:18.784122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:18.784180 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:18.784749 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:18.784761 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:18.786643 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":82,"threads_started":1}
W20260808 18:28:18.786983 21468 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:18.781362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:46201 (local address 127.7.164.126:33119)
0808 18:28:18.781617 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:18.781621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:18.781767 (+ 146us) server_negotiation.cc:407] Connection header received
0808 18:28:18.781866 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:18.781868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:18.781916 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:18.781993 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:18.782467 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:18.783005 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:18.783794 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:18.784017 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:18.784099 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:18.786821 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:46201: BlockingRecv error: recv got EOF from 127.7.164.65:46201 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":71,"threads_started":1}
Aug 08 18:28:18 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:18 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:18.979455 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:18.973609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:58093)
0808 18:28:18.973933 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:18.973947 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:18.974047 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:18.974301 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:18.974303 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:18.974507 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:18.974605 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:18.974611 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:18.975354 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:18.975356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:18.975912 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:18.975919 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:18.976450 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:18.976453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:18.976521 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:18.977047 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:18.977058 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:18.979309 (+ 2251us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":131,"threads_started":1}
W20260808 18:28:18.979645 21468 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:18.973782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:58093 (local address 127.7.164.126:33119)
0808 18:28:18.973932 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:18.973936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:18.974016 (+ 80us) server_negotiation.cc:407] Connection header received
0808 18:28:18.974131 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:18.974133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:18.974179 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:18.974277 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:18.974723 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:18.975254 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:18.976061 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:18.976305 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:18.976426 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:18.979482 (+ 3056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:58093: BlockingRecv error: recv got EOF from 127.7.164.66:58093 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 18:28:19 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:19 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:19.170097 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:19.164414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:40537)
0808 18:28:19.164700 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:19.164716 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:19.164834 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:19.165145 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:19.165148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:19.165364 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:19.165479 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:19.165486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:19.166294 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:19.166300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:19.166877 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:19.166885 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:19.167371 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:19.167375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:19.167483 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:19.168027 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:19.168039 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:19.169964 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":102,"threads_started":1}
W20260808 18:28:19.170360 21468 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:19.164644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:40537 (local address 127.7.164.126:33119)
0808 18:28:19.164783 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:19.164789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:19.164804 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:28:19.164967 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:19.164970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:19.165024 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:19.165115 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:19.165608 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:19.166151 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:19.167016 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:19.167244 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:19.167352 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:19.170175 (+ 2823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:40537: BlockingRecv error: recv got EOF from 127.7.164.67:40537 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 18:28:19 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:19 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:19.798406 21472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:19.787550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:56497)
0808 18:28:19.787835 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:19.787849 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:19.787940 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:19.788192 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:19.788194 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:19.788382 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:19.788482 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:19.788487 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:19.789227 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:19.789230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:19.789779 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:19.789787 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:19.790204 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:19.790206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:19.790336 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:19.796105 (+ 5769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:19.796120 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:19.798261 (+ 2141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":99,"threads_started":1}
W20260808 18:28:19.798704 21476 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:19.787604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:56497 (local address 127.7.164.126:33119)
0808 18:28:19.787900 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:19.787903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:19.787917 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:28:19.788023 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:19.788025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:19.788068 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:19.788151 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:19.788586 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:19.789128 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:19.789879 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:19.790109 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:19.790184 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:19.798556 (+ 8372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:56497: BlockingRecv error: recv got EOF from 127.7.164.65:56497 (error 108)
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":91,"threads_started":1}
Aug 08 18:28:19 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:19 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:19.986146 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:19.980340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:39667)
0808 18:28:19.980704 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:19.980727 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:19.980870 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:19.981151 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:19.981154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:19.981375 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:19.981472 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:19.981479 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:19.982301 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:19.982312 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:19.982896 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:19.982905 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:19.983385 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:19.983389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:19.983479 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:19.984024 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:19.984035 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:19.985956 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":97,"threads_started":1}
W20260808 18:28:19.986207 21476 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:19.980472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:39667 (local address 127.7.164.126:33119)
0808 18:28:19.980670 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:19.980675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:19.980807 (+ 132us) server_negotiation.cc:407] Connection header received
0808 18:28:19.980973 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:19.980979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:19.981034 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:19.981125 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:19.981600 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:19.982141 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:19.983031 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:19.983262 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:19.983377 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:19.986091 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:39667: BlockingRecv error: recv got EOF from 127.7.164.66:39667 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 08 18:28:20 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:20 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.67@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:20.176815 21479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:20.170955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.67:34293)
0808 18:28:20.171226 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:20.171239 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:20.171340 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:20.171857 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:20.171859 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:20.172048 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:20.172147 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:20.172152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:20.172871 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:20.172873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:20.173430 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:20.173438 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:20.173838 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:20.173840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:20.173924 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:20.174498 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:20.174510 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:20.176676 (+ 2166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":97,"threads_started":1}
W20260808 18:28:20.176892 21476 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:20.171018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:34293 (local address 127.7.164.126:33119)
0808 18:28:20.171163 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:20.171166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:20.171322 (+ 156us) server_negotiation.cc:407] Connection header received
0808 18:28:20.171689 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:20.171692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:20.171741 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:20.171814 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:20.172250 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:20.172774 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:20.173528 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:20.173742 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:20.173817 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:20.176802 (+ 2985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.67:34293: BlockingRecv error: recv got EOF from 127.7.164.67:34293 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 18:28:20 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:20 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.65@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:20.805374 21480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:20.799472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.65:53335)
0808 18:28:20.799748 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:20.799761 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:20.799856 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:20.800120 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:20.800122 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:20.800313 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:20.800414 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:20.800419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:20.801190 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:20.801193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:20.801754 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:20.801763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:20.802202 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:20.802204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:20.802364 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:20.803017 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:20.803030 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:20.805139 (+ 2109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":100,"threads_started":1}
W20260808 18:28:20.805454 21481 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:20.799508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:53335 (local address 127.7.164.126:33119)
0808 18:28:20.799761 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:20.799765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:20.799832 (+ 67us) server_negotiation.cc:407] Connection header received
0808 18:28:20.799955 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:20.799958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:20.800000 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:20.800081 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:20.800515 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:20.801065 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:20.801866 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:20.802101 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:20.802179 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:20.805291 (+ 3112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.65:53335: BlockingRecv error: recv got EOF from 127.7.164.65:53335 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":95,"threads_started":1}
Aug 08 18:28:20 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:28:20 dist-test-slave-jztv krb5kdc[19946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.164.66@KRBTEST.COM for oryx/127.7.164.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:28:20.992588 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:20.987166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33119 (local address 127.7.164.66:43505)
0808 18:28:20.987520 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:20.987534 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:20.987633 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:20.987863 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:20.987865 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:20.988059 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:20.988154 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:20.988159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:20.988895 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:20.988898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:20.989474 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:20.989482 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:20.989880 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:20.989883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:20.989952 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:20.990485 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:20.990496 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:20.992461 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.164.126:33119: Server oryx/127.7.164.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":111,"threads_started":1}
W20260808 18:28:20.992718 21481 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:20.987249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:43505 (local address 127.7.164.126:33119)
0808 18:28:20.987423 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:20.987428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:20.987630 (+ 202us) server_negotiation.cc:407] Connection header received
0808 18:28:20.987699 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:20.987701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:20.987750 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:20.987825 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:20.988255 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:20.988790 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:20.989577 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:20.989785 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:20.989861 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:20.992600 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.164.66:43505: BlockingRecv error: recv got EOF from 127.7.164.66:43505 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 18:28:21.061260 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 20806
I20260808 18:28:21.078967 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 21015
I20260808 18:28:21.103739 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 21209
I20260808 18:28:21.115307 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 20694
2026-08-08T18:28:21Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18305 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T18:28:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:28:21Z Disabled control of system clock
I20260808 18:28:21.157030 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:34537
--webserver_interface=127.7.164.126
--webserver_port=0
--builtin_ntp_servers=127.7.164.84:44933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:34537
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 18:28:21.271315 21486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:21.271595 21486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:21.271629 21486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:21.275665 21486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:28:21.275727 21486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:21.275743 21486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:21.275758 21486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:28:21.275774 21486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:28:21.280567 21486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:34537
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:34537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:21.281589 21486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:21.283025 21486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:21.291718 21497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:21.291718 21501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:21.291733 21498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:21.291778 21486 server_base.cc:1034] running on GCE node
I20260808 18:28:21.292613 21486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:21.293776 21486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:21.294960 21486 hybrid_clock.cc:648] HybridClock initialized: now 1786213701294937 us; error 33 us; skew 500 ppm
I20260808 18:28:21.296964 21486 webserver.cc:460] Webserver started at http://127.7.164.126:39865/ using document root <none> and password file <none>
I20260808 18:28:21.297600 21486 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:21.297664 21486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:21.297906 21486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:21.300149 21486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "a24ab1f349944e30b2ddfbf7c2a8796c"
format_stamp: "Formatted at 2026-08-08 18:28:21 on dist-test-slave-jztv"
server_key: "f365986b9642959b7276f85af3dedb64"
server_key_iv: "7163b38a1c903859c93593add5fcce85"
server_key_version: "encryptionkey@0"
I20260808 18:28:21.300753 21486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "a24ab1f349944e30b2ddfbf7c2a8796c"
format_stamp: "Formatted at 2026-08-08 18:28:21 on dist-test-slave-jztv"
server_key: "f365986b9642959b7276f85af3dedb64"
server_key_iv: "7163b38a1c903859c93593add5fcce85"
server_key_version: "encryptionkey@0"
I20260808 18:28:21.304586 21486 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:28:21.307067 21512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:21.308338 21486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:28:21.308513 21486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "a24ab1f349944e30b2ddfbf7c2a8796c"
format_stamp: "Formatted at 2026-08-08 18:28:21 on dist-test-slave-jztv"
server_key: "f365986b9642959b7276f85af3dedb64"
server_key_iv: "7163b38a1c903859c93593add5fcce85"
server_key_version: "encryptionkey@0"
I20260808 18:28:21.308635 21486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:21.335388 21486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:21.336164 21486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:21.336382 21486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:21.344048 21486 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:34537
I20260808 18:28:21.344182 21589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:34537 every 8 connection(s)
I20260808 18:28:21.345080 21486 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:28:21.348623 21590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:21.353300 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 21486
I20260808 18:28:21.353453 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:28:21.353753 7825 external_mini_cluster.cc:1428] Setting key d94fb241bc68bfb1585cd270d9f4f14e
I20260808 18:28:21.354691 21590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c: Bootstrap starting.
I20260808 18:28:21.357224 21590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:21.358001 21590 log.cc:824] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:28:21.359970 21590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c: No bootstrap required, opened a new log
I20260808 18:28:21.363283 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:21.355314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.7.164.126:34537)
0808 18:28:21.355999 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:21.356016 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:21.356053 (+ 37us) server_negotiation.cc:407] Connection header received
0808 18:28:21.356707 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:21.356725 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:21.357127 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:21.357464 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:21.357925 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:21.359003 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:21.359906 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:21.360433 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:21.360643 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:21.361538 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:21.361542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:21.361555 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:28:21.361933 (+ 378us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:21.361989 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:28:21.362009 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:21.362117 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:21.362309 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 18:28:21.362573 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":149,"threads_started":1}
I20260808 18:28:21.363871 21590 raft_consensus.cc:348] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a24ab1f349944e30b2ddfbf7c2a8796c" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34537 } }
I20260808 18:28:21.364127 21590 raft_consensus.cc:374] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:21.364220 21590 raft_consensus.cc:729] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a24ab1f349944e30b2ddfbf7c2a8796c, State: Initialized, Role: FOLLOWER
I20260808 18:28:21.364696 21590 consensus_queue.cc:260] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a24ab1f349944e30b2ddfbf7c2a8796c" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34537 } }
I20260808 18:28:21.364861 21590 raft_consensus.cc:388] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:28:21.364940 21590 raft_consensus.cc:482] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:28:21.365077 21590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:21.366109 21590 raft_consensus.cc:504] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a24ab1f349944e30b2ddfbf7c2a8796c" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34537 } }
I20260808 18:28:21.366468 21590 leader_election.cc:302] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a24ab1f349944e30b2ddfbf7c2a8796c; no voters:
I20260808 18:28:21.366782 21590 leader_election.cc:288] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:28:21.366904 21596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:28:21.367116 21596 raft_consensus.cc:686] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [term 1 LEADER]: Becoming Leader. State: Replica: a24ab1f349944e30b2ddfbf7c2a8796c, State: Running, Role: LEADER
I20260808 18:28:21.367491 21596 consensus_queue.cc:237] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a24ab1f349944e30b2ddfbf7c2a8796c" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34537 } }
I20260808 18:28:21.368072 21590 sys_catalog.cc:564] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:28:21.369170 21598 sys_catalog.cc:455] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a24ab1f349944e30b2ddfbf7c2a8796c. Latest consensus state: current_term: 1 leader_uuid: "a24ab1f349944e30b2ddfbf7c2a8796c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a24ab1f349944e30b2ddfbf7c2a8796c" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34537 } } }
I20260808 18:28:21.369311 21598 sys_catalog.cc:458] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [sys.catalog]: This master's current role is: LEADER
I20260808 18:28:21.369617 21597 sys_catalog.cc:455] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a24ab1f349944e30b2ddfbf7c2a8796c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a24ab1f349944e30b2ddfbf7c2a8796c" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34537 } } }
I20260808 18:28:21.369724 21597 sys_catalog.cc:458] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c [sys.catalog]: This master's current role is: LEADER
I20260808 18:28:21.369756 21605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:28:21.372349 21605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:28:21.372776 21618 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 18:28:21.377589 21605 catalog_manager.cc:1347] Generated new cluster ID: df0f13c097bb4717918009eba3e47789
I20260808 18:28:21.377657 21605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:28:21.388289 21605 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:28:21.389539 21605 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:28:21.394855 21605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a24ab1f349944e30b2ddfbf7c2a8796c: Generated new TSK 0
I20260808 18:28:21.395773 21605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:28:21.399219 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:34537
--builtin_ntp_servers=127.7.164.84:44933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 18:28:21.520147 21620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:21.520429 21620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:21.520469 21620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:21.524717 21620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:21.524780 21620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:21.524837 21620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:28:21.529938 21620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:34537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:21.531154 21620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:21.532581 21620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:21.542809 21628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:21.542794 21631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:21.542948 21620 server_base.cc:1034] running on GCE node
W20260808 18:28:21.542804 21633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:21.543577 21620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:21.544209 21620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:21.545446 21620 hybrid_clock.cc:648] HybridClock initialized: now 1786213701545409 us; error 47 us; skew 500 ppm
I20260808 18:28:21.547643 21620 webserver.cc:460] Webserver started at http://127.7.164.65:33751/ using document root <none> and password file <none>
I20260808 18:28:21.548249 21620 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:21.548346 21620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:21.548605 21620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:21.550779 21620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "1ce20d3fe596436ca266223ab46b2642"
format_stamp: "Formatted at 2026-08-08 18:28:21 on dist-test-slave-jztv"
server_key: "3dbd1b811236e6c0dc136198e8c084e5"
server_key_iv: "0e2312b5c36660bbad76f84e58d4aac6"
server_key_version: "encryptionkey@0"
I20260808 18:28:21.551338 21620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "1ce20d3fe596436ca266223ab46b2642"
format_stamp: "Formatted at 2026-08-08 18:28:21 on dist-test-slave-jztv"
server_key: "3dbd1b811236e6c0dc136198e8c084e5"
server_key_iv: "0e2312b5c36660bbad76f84e58d4aac6"
server_key_version: "encryptionkey@0"
I20260808 18:28:21.555097 21620 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:28:21.557657 21641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:21.558894 21620 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:28:21.559029 21620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "1ce20d3fe596436ca266223ab46b2642"
format_stamp: "Formatted at 2026-08-08 18:28:21 on dist-test-slave-jztv"
server_key: "3dbd1b811236e6c0dc136198e8c084e5"
server_key_iv: "0e2312b5c36660bbad76f84e58d4aac6"
server_key_version: "encryptionkey@0"
I20260808 18:28:21.559145 21620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:21.583164 21620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:21.583951 21620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:21.584167 21620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:21.584812 21620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:21.585942 21620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:21.586016 21620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:21.586083 21620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:21.586135 21620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:21.595314 21620 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:38177
I20260808 18:28:21.595345 21802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:38177 every 8 connection(s)
I20260808 18:28:21.596266 21620 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:28:21.597649 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 21620
I20260808 18:28:21.597822 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:28:21.598259 7825 external_mini_cluster.cc:1428] Setting key 179731ab381ccceaf6394bb2c2eaaecf
I20260808 18:28:21.600652 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:34537
--builtin_ntp_servers=127.7.164.84:44933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 18:28:21.607158 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:21.598336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:46537 (local address 127.7.164.126:34537)
0808 18:28:21.598485 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:21.598489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:21.600331 (+ 1842us) server_negotiation.cc:407] Connection header received
0808 18:28:21.601811 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:21.601818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:21.601886 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:21.601982 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:21.603261 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:21.603810 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:21.605007 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:21.605247 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:21.605366 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:21.606623 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:21.606629 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:21.606636 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:28:21.606650 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:21.606670 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:28:21.606678 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:21.606762 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:21.606945 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 18:28:21.607013 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:28:21.608036 21805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:21.598792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:34537 (local address 127.7.164.65:46537)
0808 18:28:21.600175 (+ 1383us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:21.600218 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:21.601584 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:21.602152 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:21.602159 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:21.602302 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:21.603093 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:21.603117 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:21.603938 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:21.603941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:21.604867 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:21.604877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:21.605413 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:21.605417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:21.605589 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:21.606308 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:28:21.606310 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:21.606353 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:28:21.606423 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:28:21.606766 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:28:21.606778 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:28:21.606785 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 18:28:21.606937 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 18:28:21.607230 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"thread_start_us":160,"threads_started":1}
I20260808 18:28:21.609349 21803 heartbeater.cc:347] Connected to a master server at 127.7.164.126:34537
I20260808 18:28:21.609632 21803 heartbeater.cc:464] Registering TS with master...
I20260808 18:28:21.610179 21803 heartbeater.cc:511] Master 127.7.164.126:34537 requested a full tablet report, sending...
I20260808 18:28:21.612030 21539 ts_manager.cc:194] Registered new tserver with Master: 1ce20d3fe596436ca266223ab46b2642 (127.7.164.65:38177)
I20260808 18:28:21.613755 21539 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.164.65:46537
W20260808 18:28:21.717934 21807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:21.718192 21807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:21.718299 21807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:21.722329 21807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:21.722383 21807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:21.722437 21807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:28:21.727252 21807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:34537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:21.728359 21807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:21.729738 21807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:21.739341 21815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:21.739348 21816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:21.739624 21807 server_base.cc:1034] running on GCE node
W20260808 18:28:21.739500 21819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:21.740188 21807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:21.740796 21807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:21.741992 21807 hybrid_clock.cc:648] HybridClock initialized: now 1786213701741969 us; error 47 us; skew 500 ppm
I20260808 18:28:21.743976 21807 webserver.cc:460] Webserver started at http://127.7.164.66:34129/ using document root <none> and password file <none>
I20260808 18:28:21.744572 21807 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:21.744670 21807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:21.744922 21807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:21.747098 21807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "826d5706b53d42409b08b416c984f238"
format_stamp: "Formatted at 2026-08-08 18:28:21 on dist-test-slave-jztv"
server_key: "9449fe4e3fa5bee6c7cc58a3f15cea90"
server_key_iv: "da285eafd6ff20b8905ecbe9de8fb0a5"
server_key_version: "encryptionkey@0"
I20260808 18:28:21.747670 21807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "826d5706b53d42409b08b416c984f238"
format_stamp: "Formatted at 2026-08-08 18:28:21 on dist-test-slave-jztv"
server_key: "9449fe4e3fa5bee6c7cc58a3f15cea90"
server_key_iv: "da285eafd6ff20b8905ecbe9de8fb0a5"
server_key_version: "encryptionkey@0"
I20260808 18:28:21.751310 21807 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:28:21.753727 21828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:21.754993 21807 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:28:21.755131 21807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "826d5706b53d42409b08b416c984f238"
format_stamp: "Formatted at 2026-08-08 18:28:21 on dist-test-slave-jztv"
server_key: "9449fe4e3fa5bee6c7cc58a3f15cea90"
server_key_iv: "da285eafd6ff20b8905ecbe9de8fb0a5"
server_key_version: "encryptionkey@0"
I20260808 18:28:21.755246 21807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:21.782437 21807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:21.783166 21807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:21.783366 21807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:21.783963 21807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:21.784936 21807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:21.785008 21807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:21.785081 21807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:21.785135 21807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:21.793694 21807 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:44125
I20260808 18:28:21.793772 21985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:44125 every 8 connection(s)
I20260808 18:28:21.794692 21807 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:28:21.798866 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 21807
I20260808 18:28:21.799014 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:28:21.799480 7825 external_mini_cluster.cc:1428] Setting key be63d464158f94ccede67289db76c0ba
I20260808 18:28:21.801561 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:34537
--builtin_ntp_servers=127.7.164.84:44933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 18:28:21.804939 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:21.796592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:46313 (local address 127.7.164.126:34537)
0808 18:28:21.796769 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:21.796774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:21.797833 (+ 1059us) server_negotiation.cc:407] Connection header received
0808 18:28:21.799795 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:21.799798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:21.799851 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:21.799968 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:21.801084 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:21.801601 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:21.802876 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:21.803106 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:21.803222 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:21.804391 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:21.804394 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:21.804396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:28:21.804408 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:21.804427 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:28:21.804435 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:21.804516 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:21.804734 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 18:28:21.804820 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 18:28:21.806115 21991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:21.797021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:34537 (local address 127.7.164.66:46313)
0808 18:28:21.797677 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:21.797719 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:21.799567 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:21.800179 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:21.800188 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:21.800295 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:21.800905 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:21.800932 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:21.801739 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:21.801743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:21.802740 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:21.802750 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:21.803267 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:21.803271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:21.803457 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:21.804110 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:28:21.804112 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:21.804153 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:28:21.804221 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:28:21.804534 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:28:21.804544 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:28:21.804551 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 18:28:21.804712 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 18:28:21.805035 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":108,"threads_started":1}
I20260808 18:28:21.807610 21986 heartbeater.cc:347] Connected to a master server at 127.7.164.126:34537
I20260808 18:28:21.807898 21986 heartbeater.cc:464] Registering TS with master...
I20260808 18:28:21.808458 21986 heartbeater.cc:511] Master 127.7.164.126:34537 requested a full tablet report, sending...
I20260808 18:28:21.809612 21539 ts_manager.cc:194] Registered new tserver with Master: 826d5706b53d42409b08b416c984f238 (127.7.164.66:44125)
I20260808 18:28:21.810256 21539 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.164.66:46313
W20260808 18:28:21.918895 21992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:21.919180 21992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:21.919260 21992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:21.923341 21992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:21.923426 21992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:21.923514 21992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:28:21.928408 21992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:34537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:21.929557 21992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:21.931038 21992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:21.940866 22000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:21.940908 21997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:21.940989 21992 server_base.cc:1034] running on GCE node
W20260808 18:28:21.940929 22002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:21.941471 21992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:21.942056 21992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:21.943215 21992 hybrid_clock.cc:648] HybridClock initialized: now 1786213701943208 us; error 34 us; skew 500 ppm
I20260808 18:28:21.945143 21992 webserver.cc:460] Webserver started at http://127.7.164.67:32991/ using document root <none> and password file <none>
I20260808 18:28:21.945700 21992 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:21.945760 21992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:21.945947 21992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:21.948051 21992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "b65cbe959cfe414f897d823bbb991058"
format_stamp: "Formatted at 2026-08-08 18:28:21 on dist-test-slave-jztv"
server_key: "6a096d07c0de9bf48112a4c7083a8fbf"
server_key_iv: "774cbcfeeb221436416afaf8f126f2af"
server_key_version: "encryptionkey@0"
I20260808 18:28:21.948591 21992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "b65cbe959cfe414f897d823bbb991058"
format_stamp: "Formatted at 2026-08-08 18:28:21 on dist-test-slave-jztv"
server_key: "6a096d07c0de9bf48112a4c7083a8fbf"
server_key_iv: "774cbcfeeb221436416afaf8f126f2af"
server_key_version: "encryptionkey@0"
I20260808 18:28:21.952450 21992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:28:21.955021 22011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:21.956230 21992 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:28:21.956326 21992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "b65cbe959cfe414f897d823bbb991058"
format_stamp: "Formatted at 2026-08-08 18:28:21 on dist-test-slave-jztv"
server_key: "6a096d07c0de9bf48112a4c7083a8fbf"
server_key_iv: "774cbcfeeb221436416afaf8f126f2af"
server_key_version: "encryptionkey@0"
I20260808 18:28:21.956427 21992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:21.976994 21992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:21.977664 21992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:21.977816 21992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:21.978392 21992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:21.979406 21992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:21.979454 21992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:21.979525 21992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:21.979563 21992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:21.988155 21992 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:45903
I20260808 18:28:21.988245 22161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:45903 every 8 connection(s)
I20260808 18:28:21.989133 21992 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:28:21.989916 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 21992
I20260808 18:28:21.990015 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:28:21.990442 7825 external_mini_cluster.cc:1428] Setting key 4023472deaf4b1deab388eed2210a595
I20260808 18:28:21.998783 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:21.991195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:55883 (local address 127.7.164.126:34537)
0808 18:28:21.991348 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:21.991354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:21.992640 (+ 1286us) server_negotiation.cc:407] Connection header received
0808 18:28:21.994063 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:21.994066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:21.994121 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:21.994200 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:21.995105 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:21.995646 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:21.996793 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:21.997028 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:21.997145 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:21.998276 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:21.998280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:21.998282 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:28:21.998295 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:21.998323 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:28:21.998331 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:21.998414 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:21.998579 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 18:28:21.998642 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:28:21.999634 22167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:21.991724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:34537 (local address 127.7.164.67:55883)
0808 18:28:21.992477 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:21.992533 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:21.993859 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:21.994396 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:21.994403 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:21.994485 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:21.994963 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:21.994975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:21.995776 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:21.995782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:21.996659 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:21.996669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:21.997186 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:21.997190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:21.997356 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:21.997997 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:28:21.997999 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:21.998038 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:28:21.998100 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:28:21.998416 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:28:21.998428 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:28:21.998438 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 18:28:21.998577 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 18:28:21.998857 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":149,"threads_started":1}
I20260808 18:28:22.000713 22162 heartbeater.cc:347] Connected to a master server at 127.7.164.126:34537
I20260808 18:28:22.000958 22162 heartbeater.cc:464] Registering TS with master...
I20260808 18:28:22.001500 22162 heartbeater.cc:511] Master 127.7.164.126:34537 requested a full tablet report, sending...
I20260808 18:28:22.002609 21539 ts_manager.cc:194] Registered new tserver with Master: b65cbe959cfe414f897d823bbb991058 (127.7.164.67:45903)
I20260808 18:28:22.003278 21539 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.164.67:55883
I20260808 18:28:22.004568 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 18:28:22.007504 21593 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:22.006499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53230 (local address 127.7.164.126:34537)
0808 18:28:22.006652 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:22.006657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:22.006865 (+ 208us) server_negotiation.cc:407] Connection header received
0808 18:28:22.006945 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:22.006947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:22.006986 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:22.007056 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:22.007376 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53230: BlockingRecv error: recv got EOF from 127.0.0.1:53230 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:28:22.007694 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 21620
I20260808 18:28:22.028579 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 21807
I20260808 18:28:22.045670 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 21992
I20260808 18:28:22.059230 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 21486
2026-08-08T18:28:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (939 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T18:28:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:28:22Z Disabled control of system clock
I20260808 18:28:22.095839 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:45987
--webserver_interface=127.7.164.126
--webserver_port=0
--builtin_ntp_servers=127.7.164.84:37377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:45987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 18:28:22.207985 22180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:22.208236 22180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:22.208273 22180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:22.213096 22180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:28:22.213171 22180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:22.213198 22180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:22.213225 22180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:28:22.213260 22180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:28:22.218107 22180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:37377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:45987
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:45987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:22.219170 22180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:22.220471 22180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:22.229059 22188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:22.229084 22191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:22.229051 22186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:22.229342 22180 server_base.cc:1034] running on GCE node
I20260808 18:28:22.229930 22180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:22.231002 22180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:22.232162 22180 hybrid_clock.cc:648] HybridClock initialized: now 1786213702232149 us; error 23 us; skew 500 ppm
I20260808 18:28:22.233943 22180 webserver.cc:460] Webserver started at http://127.7.164.126:36829/ using document root <none> and password file <none>
I20260808 18:28:22.234579 22180 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:22.234643 22180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:22.234856 22180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:22.236815 22180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "6466d87995cb4f809beefd0c4ce92972"
format_stamp: "Formatted at 2026-08-08 18:28:22 on dist-test-slave-jztv"
server_key: "8969bdbcb1f3e296e87afe9108cb3b0a"
server_key_iv: "e6a62e1779a050ab5fa8dfe98453ca54"
server_key_version: "encryptionkey@0"
I20260808 18:28:22.237375 22180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "6466d87995cb4f809beefd0c4ce92972"
format_stamp: "Formatted at 2026-08-08 18:28:22 on dist-test-slave-jztv"
server_key: "8969bdbcb1f3e296e87afe9108cb3b0a"
server_key_iv: "e6a62e1779a050ab5fa8dfe98453ca54"
server_key_version: "encryptionkey@0"
I20260808 18:28:22.241039 22180 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 18:28:22.243566 22200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:22.244802 22180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:28:22.244936 22180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "6466d87995cb4f809beefd0c4ce92972"
format_stamp: "Formatted at 2026-08-08 18:28:22 on dist-test-slave-jztv"
server_key: "8969bdbcb1f3e296e87afe9108cb3b0a"
server_key_iv: "e6a62e1779a050ab5fa8dfe98453ca54"
server_key_version: "encryptionkey@0"
I20260808 18:28:22.245049 22180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:22.249706 22180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:22.250382 22180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:22.250568 22180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:22.257531 22267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:45987 every 8 connection(s)
I20260808 18:28:22.257531 22180 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:45987
I20260808 18:28:22.258570 22180 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:28:22.261865 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 22180
I20260808 18:28:22.261978 22270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:22.261992 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:28:22.262377 7825 external_mini_cluster.cc:1428] Setting key a34397969bd9c8bcc250d4bb22e11120
I20260808 18:28:22.267966 22273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:22.263846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46048 (local address 127.7.164.126:45987)
0808 18:28:22.264483 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:22.264498 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:22.264542 (+ 44us) server_negotiation.cc:407] Connection header received
0808 18:28:22.265250 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:22.265267 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:22.265611 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:22.265942 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:22.266759 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:22.266765 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:22.266785 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:28:22.266795 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:22.266847 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:28:22.266864 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:22.267016 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:22.267086 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 18:28:22.267321 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":135,"threads_started":1}
I20260808 18:28:22.268746 22270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972: Bootstrap starting.
I20260808 18:28:22.271029 22270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:22.271765 22270 log.cc:824] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972: Log is configured to *not* fsync() on all Append() calls
I20260808 18:28:22.273561 22270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972: No bootstrap required, opened a new log
I20260808 18:28:22.277311 22270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6466d87995cb4f809beefd0c4ce92972" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45987 } }
I20260808 18:28:22.277560 22270 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:22.277644 22270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6466d87995cb4f809beefd0c4ce92972, State: Initialized, Role: FOLLOWER
I20260808 18:28:22.278057 22270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6466d87995cb4f809beefd0c4ce92972" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45987 } }
I20260808 18:28:22.278213 22270 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:28:22.278313 22270 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:28:22.278396 22270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:22.279459 22270 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6466d87995cb4f809beefd0c4ce92972" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45987 } }
I20260808 18:28:22.279832 22270 leader_election.cc:302] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6466d87995cb4f809beefd0c4ce92972; no voters:
I20260808 18:28:22.280135 22270 leader_election.cc:288] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:28:22.280265 22276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:28:22.280511 22276 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [term 1 LEADER]: Becoming Leader. State: Replica: 6466d87995cb4f809beefd0c4ce92972, State: Running, Role: LEADER
I20260808 18:28:22.280860 22276 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6466d87995cb4f809beefd0c4ce92972" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45987 } }
I20260808 18:28:22.281548 22270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:28:22.282164 22276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6466d87995cb4f809beefd0c4ce92972. Latest consensus state: current_term: 1 leader_uuid: "6466d87995cb4f809beefd0c4ce92972" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6466d87995cb4f809beefd0c4ce92972" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45987 } } }
I20260808 18:28:22.282325 22276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [sys.catalog]: This master's current role is: LEADER
I20260808 18:28:22.283332 22277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6466d87995cb4f809beefd0c4ce92972" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6466d87995cb4f809beefd0c4ce92972" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45987 } } }
I20260808 18:28:22.283505 22284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:28:22.283612 22277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972 [sys.catalog]: This master's current role is: LEADER
I20260808 18:28:22.287101 22284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:28:22.298374 22284 catalog_manager.cc:1347] Generated new cluster ID: 75ff1e62eab9426aa28e56da0290a720
I20260808 18:28:22.298465 22284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:28:22.308673 22284 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:28:22.309010 22284 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:28:22.319504 22284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6466d87995cb4f809beefd0c4ce92972: Generated new TSK 0
I20260808 18:28:22.320242 22284 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:28:22.325269 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45987
--builtin_ntp_servers=127.7.164.84:37377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 18:28:22.461176 22302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:22.461458 22302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:22.461500 22302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:22.465691 22302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:22.465764 22302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:22.465823 22302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:28:22.470741 22302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:37377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:22.471832 22302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:22.473270 22302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:22.482740 22311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:22.482746 22312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:22.482919 22314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:22.483095 22302 server_base.cc:1034] running on GCE node
I20260808 18:28:22.483655 22302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:22.484359 22302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:22.485563 22302 hybrid_clock.cc:648] HybridClock initialized: now 1786213702485530 us; error 48 us; skew 500 ppm
I20260808 18:28:22.487553 22302 webserver.cc:460] Webserver started at http://127.7.164.65:45889/ using document root <none> and password file <none>
I20260808 18:28:22.488147 22302 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:22.488243 22302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:22.488490 22302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:22.490624 22302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "024e4594ffb941bca823c7e016789958"
format_stamp: "Formatted at 2026-08-08 18:28:22 on dist-test-slave-jztv"
server_key: "b06cf549680465f61921bb8d2bf67bcb"
server_key_iv: "d04d299d92e754f070091ca272e7f861"
server_key_version: "encryptionkey@0"
I20260808 18:28:22.491226 22302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "024e4594ffb941bca823c7e016789958"
format_stamp: "Formatted at 2026-08-08 18:28:22 on dist-test-slave-jztv"
server_key: "b06cf549680465f61921bb8d2bf67bcb"
server_key_iv: "d04d299d92e754f070091ca272e7f861"
server_key_version: "encryptionkey@0"
I20260808 18:28:22.495278 22302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 18:28:22.497910 22322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:22.499205 22302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 18:28:22.499352 22302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "024e4594ffb941bca823c7e016789958"
format_stamp: "Formatted at 2026-08-08 18:28:22 on dist-test-slave-jztv"
server_key: "b06cf549680465f61921bb8d2bf67bcb"
server_key_iv: "d04d299d92e754f070091ca272e7f861"
server_key_version: "encryptionkey@0"
I20260808 18:28:22.499480 22302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:22.504194 22302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:22.504797 22302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:22.504987 22302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:22.505518 22302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:22.506484 22302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:22.506552 22302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:22.506623 22302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:22.506666 22302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:22.515676 22302 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:34891
I20260808 18:28:22.515704 22488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:34891 every 8 connection(s)
I20260808 18:28:22.516582 22302 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:28:22.523093 22273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:22.518491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:32769 (local address 127.7.164.126:45987)
0808 18:28:22.518654 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:22.518658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:22.519794 (+ 1136us) server_negotiation.cc:407] Connection header received
0808 18:28:22.521182 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:22.521185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:22.521231 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:22.521324 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:22.522582 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:22.522586 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:22.522589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:28:22.522602 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:22.522625 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:28:22.522642 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:22.522747 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:22.522906 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 18:28:22.522955 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 18:28:22.523916 22491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:22.518871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45987 (local address 127.7.164.65:32769)
0808 18:28:22.519630 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:22.519673 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:22.520961 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:22.521515 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:22.521522 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:22.521620 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:22.522304 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:28:22.522306 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:22.522356 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:28:22.522427 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:28:22.522729 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:28:22.522739 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:28:22.522745 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:28:22.522937 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 18:28:22.523177 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":112,"threads_started":1}
I20260808 18:28:22.524106 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 22302
I20260808 18:28:22.524263 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:28:22.524645 7825 external_mini_cluster.cc:1428] Setting key 9a46df63422e4fdc330b91a701dc51e1
I20260808 18:28:22.525365 22489 heartbeater.cc:347] Connected to a master server at 127.7.164.126:45987
I20260808 18:28:22.525612 22489 heartbeater.cc:464] Registering TS with master...
I20260808 18:28:22.526110 22489 heartbeater.cc:511] Master 127.7.164.126:45987 requested a full tablet report, sending...
I20260808 18:28:22.526942 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45987
--builtin_ntp_servers=127.7.164.84:37377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 18:28:22.527794 22222 ts_manager.cc:194] Registered new tserver with Master: 024e4594ffb941bca823c7e016789958 (127.7.164.65:34891)
W20260808 18:28:22.643306 22493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:22.643591 22493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:22.643631 22493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:22.647714 22493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:22.647776 22493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:22.647835 22493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:28:22.652614 22493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:37377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:22.653769 22493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:22.655275 22493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:22.665676 22500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:22.665692 22504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:22.665692 22499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:22.666011 22493 server_base.cc:1034] running on GCE node
I20260808 18:28:22.666522 22493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:22.667184 22493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:22.668375 22493 hybrid_clock.cc:648] HybridClock initialized: now 1786213702668356 us; error 39 us; skew 500 ppm
I20260808 18:28:22.670467 22493 webserver.cc:460] Webserver started at http://127.7.164.66:43883/ using document root <none> and password file <none>
I20260808 18:28:22.671092 22493 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:22.671190 22493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:22.671439 22493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:22.673607 22493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "216ae37103174aac92b5ea451ab71fa6"
format_stamp: "Formatted at 2026-08-08 18:28:22 on dist-test-slave-jztv"
server_key: "93c936999a117374d1e8cd399082be43"
server_key_iv: "2f54e7c9a75e74f03b1863ffc4ccf0d9"
server_key_version: "encryptionkey@0"
I20260808 18:28:22.674612 22493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "216ae37103174aac92b5ea451ab71fa6"
format_stamp: "Formatted at 2026-08-08 18:28:22 on dist-test-slave-jztv"
server_key: "93c936999a117374d1e8cd399082be43"
server_key_iv: "2f54e7c9a75e74f03b1863ffc4ccf0d9"
server_key_version: "encryptionkey@0"
I20260808 18:28:22.678625 22493 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 18:28:22.681365 22514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:22.682796 22493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:28:22.682991 22493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "216ae37103174aac92b5ea451ab71fa6"
format_stamp: "Formatted at 2026-08-08 18:28:22 on dist-test-slave-jztv"
server_key: "93c936999a117374d1e8cd399082be43"
server_key_iv: "2f54e7c9a75e74f03b1863ffc4ccf0d9"
server_key_version: "encryptionkey@0"
I20260808 18:28:22.683110 22493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:22.688060 22493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:22.688684 22493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:22.688875 22493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:22.689440 22493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:22.690459 22493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:22.690508 22493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:22.690572 22493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:22.690605 22493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:22.699730 22493 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:38285
I20260808 18:28:22.699753 22667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:38285 every 8 connection(s)
I20260808 18:28:22.700757 22493 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:28:22.704697 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 22493
I20260808 18:28:22.704870 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:28:22.705174 7825 external_mini_cluster.cc:1428] Setting key b9e31cb3b03b595efbc2e713baa89469
I20260808 18:28:22.707479 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45987
--builtin_ntp_servers=127.7.164.84:37377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 18:28:22.707489 22273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:22.702797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:42573 (local address 127.7.164.126:45987)
0808 18:28:22.702945 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:22.702949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:22.704123 (+ 1174us) server_negotiation.cc:407] Connection header received
0808 18:28:22.705600 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:22.705603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:22.705643 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:22.705720 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:22.707046 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:22.707050 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:22.707052 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:28:22.707063 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:22.707082 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:28:22.707090 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:22.707165 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:22.707342 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 18:28:22.707379 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:28:22.708649 22672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:22.703218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45987 (local address 127.7.164.66:42573)
0808 18:28:22.703965 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:22.704006 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:22.705370 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:22.705925 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:22.705932 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:22.706031 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:22.706734 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:28:22.706736 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:22.706792 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:28:22.706878 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:28:22.707182 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:28:22.707195 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:28:22.707202 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 18:28:22.707386 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 18:28:22.707645 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":153,"threads_started":1}
I20260808 18:28:22.709884 22668 heartbeater.cc:347] Connected to a master server at 127.7.164.126:45987
I20260808 18:28:22.710150 22668 heartbeater.cc:464] Registering TS with master...
I20260808 18:28:22.710600 22668 heartbeater.cc:511] Master 127.7.164.126:45987 requested a full tablet report, sending...
I20260808 18:28:22.711726 22222 ts_manager.cc:194] Registered new tserver with Master: 216ae37103174aac92b5ea451ab71fa6 (127.7.164.66:38285)
W20260808 18:28:22.824788 22675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:22.825055 22675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:22.825095 22675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:22.829152 22675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:22.829236 22675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:22.829291 22675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:28:22.833951 22675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:37377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:22.835008 22675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:22.836323 22675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:22.845770 22681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:22.845866 22675 server_base.cc:1034] running on GCE node
W20260808 18:28:22.845816 22685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:22.845777 22683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:22.846558 22675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:22.847172 22675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:22.848340 22675 hybrid_clock.cc:648] HybridClock initialized: now 1786213702848301 us; error 47 us; skew 500 ppm
I20260808 18:28:22.850373 22675 webserver.cc:460] Webserver started at http://127.7.164.67:33763/ using document root <none> and password file <none>
I20260808 18:28:22.850951 22675 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:22.851015 22675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:22.851274 22675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:22.853536 22675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "6932f8621319409ea769c2bf42bde26b"
format_stamp: "Formatted at 2026-08-08 18:28:22 on dist-test-slave-jztv"
server_key: "ab5b482230c4a9370ff968dd5b311e3d"
server_key_iv: "9243a2f2ad450640e93b4ba557741c02"
server_key_version: "encryptionkey@0"
I20260808 18:28:22.854120 22675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "6932f8621319409ea769c2bf42bde26b"
format_stamp: "Formatted at 2026-08-08 18:28:22 on dist-test-slave-jztv"
server_key: "ab5b482230c4a9370ff968dd5b311e3d"
server_key_iv: "9243a2f2ad450640e93b4ba557741c02"
server_key_version: "encryptionkey@0"
I20260808 18:28:22.857867 22675 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:28:22.860409 22692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:22.861737 22675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:28:22.861879 22675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "6932f8621319409ea769c2bf42bde26b"
format_stamp: "Formatted at 2026-08-08 18:28:22 on dist-test-slave-jztv"
server_key: "ab5b482230c4a9370ff968dd5b311e3d"
server_key_iv: "9243a2f2ad450640e93b4ba557741c02"
server_key_version: "encryptionkey@0"
I20260808 18:28:22.861995 22675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:22.866504 22675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:22.867086 22675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:22.867282 22675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:22.867812 22675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:22.868709 22675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:22.868781 22675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:22.868844 22675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:22.868893 22675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:22.877457 22675 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:42219
I20260808 18:28:22.877535 22855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:42219 every 8 connection(s)
I20260808 18:28:22.878407 22675 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:28:22.885005 22273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:22.880457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:55947 (local address 127.7.164.126:45987)
0808 18:28:22.880607 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:22.880611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:22.881604 (+ 993us) server_negotiation.cc:407] Connection header received
0808 18:28:22.883138 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:22.883141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:22.883187 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:22.883298 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:22.884449 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:22.884452 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:22.884455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:28:22.884472 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:22.884503 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:28:22.884512 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:22.884625 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:22.884803 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 18:28:22.884855 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:28:22.885921 22864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:22.880822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45987 (local address 127.7.164.67:55947)
0808 18:28:22.881456 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:22.881495 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:22.882899 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:22.883479 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:22.883485 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:22.883573 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:22.884134 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:28:22.884136 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:22.884179 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:28:22.884254 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:28:22.884609 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:28:22.884620 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:28:22.884626 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:28:22.884847 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 18:28:22.885110 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":117,"threads_started":1}
I20260808 18:28:22.886901 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 22675
I20260808 18:28:22.887003 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:28:22.887290 22856 heartbeater.cc:347] Connected to a master server at 127.7.164.126:45987
I20260808 18:28:22.887382 7825 external_mini_cluster.cc:1428] Setting key 817162081aee831d25d342f7711b3417
I20260808 18:28:22.887550 22856 heartbeater.cc:464] Registering TS with master...
I20260808 18:28:22.888067 22856 heartbeater.cc:511] Master 127.7.164.126:45987 requested a full tablet report, sending...
I20260808 18:28:22.889246 22219 ts_manager.cc:194] Registered new tserver with Master: 6932f8621319409ea769c2bf42bde26b (127.7.164.67:42219)
I20260808 18:28:22.901262 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 18:28:22.904433 22273 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:28:22.903385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46062 (local address 127.7.164.126:45987)
0808 18:28:22.903522 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:22.903528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:22.903740 (+ 212us) server_negotiation.cc:407] Connection header received
0808 18:28:22.903913 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:22.903916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:22.903952 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:22.904040 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:22.904322 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46062: BlockingRecv error: recv got EOF from 127.0.0.1:46062 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 18:28:22.904772 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 22302
I20260808 18:28:22.918782 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 22493
I20260808 18:28:22.933816 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 22675
I20260808 18:28:22.941340 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 22180
2026-08-08T18:28:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (882 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:28:23 dist-test-slave-jztv krb5kdc[22883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:28:23 dist-test-slave-jztv krb5kdc[22883](info): set up 2 sockets
Aug 08 18:28:23 dist-test-slave-jztv krb5kdc[22883](info): commencing operation
krb5kdc: starting...
W20260808 18:28:24.164166 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:28:24 dist-test-slave-jztv krb5kdc[22883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:28:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:28:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:28:24.297699 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:44229
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:46553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:44229
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:28:24.409889 22901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:24.410205 22901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:24.410305 22901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:24.414315 22901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:28:24.414413 22901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:24.414453 22901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:24.414513 22901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:28:24.414552 22901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:28:24.419835 22901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:46553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:44229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:44229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:24.420958 22901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:24.422015 22901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:28:24.431079 22901 server_base.cc:1034] running on GCE node
W20260808 18:28:24.431079 22912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:24.431052 22910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:24.431097 22909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:24.431908 22901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:24.433192 22901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:24.434394 22901 hybrid_clock.cc:648] HybridClock initialized: now 1786213704434383 us; error 36 us; skew 500 ppm
Aug 08 18:28:24 dist-test-slave-jztv krb5kdc[22883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213704, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:28:24.437348 22901 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:28:24.438304 22901 webserver.cc:460] Webserver started at http://127.7.164.126:39303/ using document root <none> and password file <none>
I20260808 18:28:24.438880 22901 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:24.438973 22901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:24.439224 22901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:24.441310 22901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "a64de437a6244eb1a1808fceb9517e10"
format_stamp: "Formatted at 2026-08-08 18:28:24 on dist-test-slave-jztv"
server_key: "9bf5f4161098c1505541dbb9f42c5008"
server_key_iv: "6b39da71f9cfed45f8001fb7f1f239b9"
server_key_version: "encryptionkey@0"
I20260808 18:28:24.441895 22901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "a64de437a6244eb1a1808fceb9517e10"
format_stamp: "Formatted at 2026-08-08 18:28:24 on dist-test-slave-jztv"
server_key: "9bf5f4161098c1505541dbb9f42c5008"
server_key_iv: "6b39da71f9cfed45f8001fb7f1f239b9"
server_key_version: "encryptionkey@0"
I20260808 18:28:24.445713 22901 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:28:24.448168 22922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:24.449342 22901 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:28:24.449496 22901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "a64de437a6244eb1a1808fceb9517e10"
format_stamp: "Formatted at 2026-08-08 18:28:24 on dist-test-slave-jztv"
server_key: "9bf5f4161098c1505541dbb9f42c5008"
server_key_iv: "6b39da71f9cfed45f8001fb7f1f239b9"
server_key_version: "encryptionkey@0"
I20260808 18:28:24.449616 22901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:24.476034 22901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:24.479429 22901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:24.479638 22901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:24.486851 22901 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:44229
I20260808 18:28:24.487152 22989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:44229 every 8 connection(s)
I20260808 18:28:24.487854 22901 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:28:24.490561 22991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:24.494848 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 22901
I20260808 18:28:24.494948 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:28:24.495242 7825 external_mini_cluster.cc:1428] Setting key b1dfde3c3ab2eb7a7f6bf193de067a22
I20260808 18:28:24.496575 22991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10: Bootstrap starting.
I20260808 18:28:24.499409 22991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:24.500152 22991 log.cc:824] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10: Log is configured to *not* fsync() on all Append() calls
I20260808 18:28:24.502302 22991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10: No bootstrap required, opened a new log
Aug 08 18:28:24 dist-test-slave-jztv krb5kdc[22883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:28:24.506632 22991 raft_consensus.cc:348] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64de437a6244eb1a1808fceb9517e10" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44229 } }
I20260808 18:28:24.506922 22991 raft_consensus.cc:374] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:24.506990 22991 raft_consensus.cc:729] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a64de437a6244eb1a1808fceb9517e10, State: Initialized, Role: FOLLOWER
I20260808 18:28:24.507524 22991 consensus_queue.cc:260] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64de437a6244eb1a1808fceb9517e10" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44229 } }
I20260808 18:28:24.507696 22991 raft_consensus.cc:388] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:28:24.507766 22991 raft_consensus.cc:482] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:28:24.507866 22991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:24.509038 22991 raft_consensus.cc:504] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64de437a6244eb1a1808fceb9517e10" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44229 } }
I20260808 18:28:24.509399 22991 leader_election.cc:302] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a64de437a6244eb1a1808fceb9517e10; no voters:
I20260808 18:28:24.509727 22991 leader_election.cc:288] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:28:24.509845 23000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:28:24.510118 23000 raft_consensus.cc:686] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [term 1 LEADER]: Becoming Leader. State: Replica: a64de437a6244eb1a1808fceb9517e10, State: Running, Role: LEADER
I20260808 18:28:24.510306 22995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:24.496679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44682 (local address 127.7.164.126:44229)
0808 18:28:24.497137 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:24.497158 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:24.497201 (+ 43us) server_negotiation.cc:407] Connection header received
0808 18:28:24.498069 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:24.498093 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:24.498545 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:24.498897 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:24.499650 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:24.500688 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:24.501606 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:24.502281 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:24.502447 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:24.504906 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:24.504915 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:24.504936 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:28:24.504978 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:24.506981 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:24.507419 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:24.507424 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:24.507430 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:24.507583 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:24.507987 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:24.507991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:24.507993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:24.508503 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:24.508712 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:24.508945 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 18:28:24.509256 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":105,"threads_started":1}
I20260808 18:28:24.510473 23000 consensus_queue.cc:237] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64de437a6244eb1a1808fceb9517e10" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44229 } }
I20260808 18:28:24.511027 22991 sys_catalog.cc:564] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:28:24.512054 23001 sys_catalog.cc:455] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a64de437a6244eb1a1808fceb9517e10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64de437a6244eb1a1808fceb9517e10" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44229 } } }
I20260808 18:28:24.512177 23001 sys_catalog.cc:458] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [sys.catalog]: This master's current role is: LEADER
I20260808 18:28:24.512487 23002 sys_catalog.cc:455] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a64de437a6244eb1a1808fceb9517e10. Latest consensus state: current_term: 1 leader_uuid: "a64de437a6244eb1a1808fceb9517e10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a64de437a6244eb1a1808fceb9517e10" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44229 } } }
I20260808 18:28:24.512571 23002 sys_catalog.cc:458] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10 [sys.catalog]: This master's current role is: LEADER
I20260808 18:28:24.512857 23014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:28:24.516620 23014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:28:24.522068 23014 catalog_manager.cc:1347] Generated new cluster ID: 66afc97151d941a887673a7185477b42
I20260808 18:28:24.522148 23014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:28:24.551731 23014 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:28:24.553025 23014 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:28:24.559190 23014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10: Generated new TSK 0
I20260808 18:28:24.559890 23014 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:28:24.596998 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:44229
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:46553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:28:24.735762 23030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:24.736003 23030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:24.736045 23030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:24.739885 23030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:24.739930 23030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:24.739989 23030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:28:24.744932 23030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:46553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:44229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:24.745898 23030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:24.746872 23030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:24.756518 23041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:24.756580 23037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:24.756592 23039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:24.756703 23030 server_base.cc:1034] running on GCE node
I20260808 18:28:24.757241 23030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:24.757946 23030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:24.759135 23030 hybrid_clock.cc:648] HybridClock initialized: now 1786213704759111 us; error 36 us; skew 500 ppm
Aug 08 18:28:24 dist-test-slave-jztv krb5kdc[22883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213704, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:28:24.761862 23030 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:28:24.762843 23030 webserver.cc:460] Webserver started at http://127.7.164.65:32871/ using document root <none> and password file <none>
I20260808 18:28:24.763414 23030 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:24.763515 23030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:24.763756 23030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:24.765812 23030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "e8f1ab3085bd46e2a108e3deb4a28030"
format_stamp: "Formatted at 2026-08-08 18:28:24 on dist-test-slave-jztv"
server_key: "b2ab5068b89dcb3368ee8b681e150fef"
server_key_iv: "7e9926dc19418b78f2a9a0d0e195be27"
server_key_version: "encryptionkey@0"
I20260808 18:28:24.766446 23030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "e8f1ab3085bd46e2a108e3deb4a28030"
format_stamp: "Formatted at 2026-08-08 18:28:24 on dist-test-slave-jztv"
server_key: "b2ab5068b89dcb3368ee8b681e150fef"
server_key_iv: "7e9926dc19418b78f2a9a0d0e195be27"
server_key_version: "encryptionkey@0"
I20260808 18:28:24.770067 23030 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:28:24.772490 23051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:24.773725 23030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:28:24.773864 23030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "e8f1ab3085bd46e2a108e3deb4a28030"
format_stamp: "Formatted at 2026-08-08 18:28:24 on dist-test-slave-jztv"
server_key: "b2ab5068b89dcb3368ee8b681e150fef"
server_key_iv: "7e9926dc19418b78f2a9a0d0e195be27"
server_key_version: "encryptionkey@0"
I20260808 18:28:24.773975 23030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:24.804330 23030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:24.807224 23030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:24.807433 23030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:24.808033 23030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:24.808866 23030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:24.808938 23030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:24.809005 23030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:24.809059 23030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:24.818257 23030 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:42887
I20260808 18:28:24.818288 23215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:42887 every 8 connection(s)
I20260808 18:28:24.819115 23030 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:28:24.825871 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 23030
I20260808 18:28:24.825989 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:28:24.826337 7825 external_mini_cluster.cc:1428] Setting key 98817a4292b7e11942c4a142343f25c5
Aug 08 18:28:24 dist-test-slave-jztv krb5kdc[22883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213704, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:28:24.833541 22995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:24.820862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:54121 (local address 127.7.164.126:44229)
0808 18:28:24.821012 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:24.821017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:24.821900 (+ 883us) server_negotiation.cc:407] Connection header received
0808 18:28:24.823268 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:24.823271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:24.823318 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:24.823391 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:24.824459 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:24.824985 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:24.826411 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:24.826669 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:24.826795 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:24.829638 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:24.829642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:24.829645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:28:24.829678 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:24.831453 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:24.831996 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:24.831999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:24.832001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:24.832092 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:24.832439 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:24.832444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:24.832445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:24.832633 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:24.832744 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:24.833234 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 18:28:24.833371 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:28:24.834393 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:24.821297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:44229 (local address 127.7.164.65:54121)
0808 18:28:24.821747 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:24.821791 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:24.823080 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:24.823524 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:24.823530 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:24.823869 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:24.824332 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:24.824346 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:24.825094 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:24.825096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:24.826280 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:24.826292 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:24.826838 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:24.826841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:24.827045 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:24.827752 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:24.827771 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:24.829469 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:28:24.831582 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:28:24.831588 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:28:24.831603 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:28:24.831874 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:28:24.832187 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:28:24.832190 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:28:24.832192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:28:24.832335 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:28:24.832744 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:28:24.832752 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:28:24.832991 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 18:28:24.833176 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 18:28:24.833490 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":112,"threads_started":1}
I20260808 18:28:24.835795 23216 heartbeater.cc:347] Connected to a master server at 127.7.164.126:44229
I20260808 18:28:24.836064 23216 heartbeater.cc:464] Registering TS with master...
I20260808 18:28:24.836588 23216 heartbeater.cc:511] Master 127.7.164.126:44229 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:28:24.838335 22946 ts_manager.cc:194] Registered new tserver with Master: e8f1ab3085bd46e2a108e3deb4a28030 (127.7.164.65:42887)
I20260808 18:28:24.839500 22946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:54121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:28:24.862052 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:44229
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:46553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:28:24.975873 23221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:24.976146 23221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:24.976189 23221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:24.980247 23221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:24.980302 23221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:24.980360 23221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:28:24.985456 23221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:46553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:44229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:24.986436 23221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:24.987413 23221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:24.997102 23226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:24.997098 23229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:24.997237 23221 server_base.cc:1034] running on GCE node
W20260808 18:28:24.997375 23227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:24.997817 23221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:24.998492 23221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:24.999655 23221 hybrid_clock.cc:648] HybridClock initialized: now 1786213704999620 us; error 48 us; skew 500 ppm
Aug 08 18:28:24 dist-test-slave-jztv krb5kdc[22883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213704, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:28:25.002555 23221 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:28:25.003486 23221 webserver.cc:460] Webserver started at http://127.7.164.66:44377/ using document root <none> and password file <none>
I20260808 18:28:25.004061 23221 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:25.004156 23221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:25.004410 23221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:25.006544 23221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "d60641e02af74d969c10527766a89dff"
format_stamp: "Formatted at 2026-08-08 18:28:25 on dist-test-slave-jztv"
server_key: "5681be7d75cef581580c5afb6b547d02"
server_key_iv: "d646b75e9947438cf2e66aa06aae66e9"
server_key_version: "encryptionkey@0"
I20260808 18:28:25.007148 23221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "d60641e02af74d969c10527766a89dff"
format_stamp: "Formatted at 2026-08-08 18:28:25 on dist-test-slave-jztv"
server_key: "5681be7d75cef581580c5afb6b547d02"
server_key_iv: "d646b75e9947438cf2e66aa06aae66e9"
server_key_version: "encryptionkey@0"
I20260808 18:28:25.011166 23221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:28:25.014002 23242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:25.015339 23221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:28:25.015494 23221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "d60641e02af74d969c10527766a89dff"
format_stamp: "Formatted at 2026-08-08 18:28:25 on dist-test-slave-jztv"
server_key: "5681be7d75cef581580c5afb6b547d02"
server_key_iv: "d646b75e9947438cf2e66aa06aae66e9"
server_key_version: "encryptionkey@0"
I20260808 18:28:25.015621 23221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:25.031066 23221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:25.034114 23221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:25.034346 23221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:25.034967 23221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:25.035830 23221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:25.035903 23221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:25.035976 23221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:25.036029 23221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:25.045372 23221 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:45731
I20260808 18:28:25.045393 23399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:45731 every 8 connection(s)
I20260808 18:28:25.046357 23221 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:28:25.049137 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 23221
I20260808 18:28:25.049261 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:28:25.049624 7825 external_mini_cluster.cc:1428] Setting key 7cab94575fe4dfab722670d1417e5728
Aug 08 18:28:25 dist-test-slave-jztv krb5kdc[22883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213704, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
I20260808 18:28:25.062281 22995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.048251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:46505 (local address 127.7.164.126:44229)
0808 18:28:25.048391 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:25.048395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:25.049315 (+ 920us) server_negotiation.cc:407] Connection header received
0808 18:28:25.051144 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:25.051148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:25.051199 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:25.051278 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:25.052462 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.053087 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.054436 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.054833 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.054960 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.057817 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:25.057823 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:25.057829 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:28:25.057867 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:25.059912 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:25.060521 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:25.060528 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:25.060531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:25.060633 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:25.061023 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:25.061029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:25.061032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:25.061311 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:25.061422 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:25.061942 (+ 520us) server_negotiation.cc:299] Negotiation successful
0808 18:28:25.062059 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:28:25.063143 23402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.048584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:44229 (local address 127.7.164.66:46505)
0808 18:28:25.049136 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:25.049197 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:25.050914 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:25.051450 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:25.051457 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:25.051812 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:25.052299 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:25.052312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.053216 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.053219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:25.054314 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:25.054325 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.055001 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.055004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:25.055164 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.055793 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:25.055816 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:25.057614 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:28:25.060070 (+ 2456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:28:25.060075 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:28:25.060090 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:28:25.060392 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:28:25.060761 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:28:25.060764 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:28:25.060765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:28:25.060918 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:28:25.061437 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:28:25.061442 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:28:25.061704 (+ 262us) client_negotiation.cc:769] Sending connection context
0808 18:28:25.061872 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 18:28:25.062149 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":117,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
I20260808 18:28:25.064528 23400 heartbeater.cc:347] Connected to a master server at 127.7.164.126:44229
I20260808 18:28:25.064826 23400 heartbeater.cc:464] Registering TS with master...
I20260808 18:28:25.065465 23400 heartbeater.cc:511] Master 127.7.164.126:44229 requested a full tablet report, sending...
I20260808 18:28:25.066811 22946 ts_manager.cc:194] Registered new tserver with Master: d60641e02af74d969c10527766a89dff (127.7.164.66:45731)
I20260808 18:28:25.067679 22946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:46505
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:28:25.082171 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:44229
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:46553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:28:25.196718 23405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:28:25.196970 23405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:28:25.197013 23405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:28:25.201049 23405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:28:25.201112 23405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:28:25.201171 23405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:28:25.206514 23405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:46553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:44229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:28:25.207561 23405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:28:25.208563 23405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:28:25.218533 23413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:25.218479 23416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:28:25.218542 23411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:28:25.218724 23405 server_base.cc:1034] running on GCE node
I20260808 18:28:25.219297 23405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:28:25.220019 23405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:28:25.221220 23405 hybrid_clock.cc:648] HybridClock initialized: now 1786213705221208 us; error 31 us; skew 500 ppm
Aug 08 18:28:25 dist-test-slave-jztv krb5kdc[22883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213705, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:28:25.225054 23405 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:28:25.226073 23405 webserver.cc:460] Webserver started at http://127.7.164.67:44475/ using document root <none> and password file <none>
I20260808 18:28:25.226727 23405 fs_manager.cc:359] Metadata directory not provided
I20260808 18:28:25.226845 23405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:28:25.227085 23405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:28:25.229280 23405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "7234089f754d47cf954f0a781d7b6105"
format_stamp: "Formatted at 2026-08-08 18:28:25 on dist-test-slave-jztv"
server_key: "325a54c106177a54c8ed7d626513e02a"
server_key_iv: "1262d4203c6577dd23468805ae49f120"
server_key_version: "encryptionkey@0"
I20260808 18:28:25.229928 23405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "7234089f754d47cf954f0a781d7b6105"
format_stamp: "Formatted at 2026-08-08 18:28:25 on dist-test-slave-jztv"
server_key: "325a54c106177a54c8ed7d626513e02a"
server_key_iv: "1262d4203c6577dd23468805ae49f120"
server_key_version: "encryptionkey@0"
I20260808 18:28:25.234115 23405 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:28:25.236594 23424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:25.237782 23405 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:28:25.237923 23405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "7234089f754d47cf954f0a781d7b6105"
format_stamp: "Formatted at 2026-08-08 18:28:25 on dist-test-slave-jztv"
server_key: "325a54c106177a54c8ed7d626513e02a"
server_key_iv: "1262d4203c6577dd23468805ae49f120"
server_key_version: "encryptionkey@0"
I20260808 18:28:25.238049 23405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:28:25.249153 23405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:28:25.252792 23405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:28:25.253018 23405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:28:25.253721 23405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:28:25.254660 23405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:28:25.254734 23405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:25.254806 23405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:28:25.254850 23405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:28:25.263746 23405 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:38299
I20260808 18:28:25.263765 23593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:38299 every 8 connection(s)
I20260808 18:28:25.264627 23405 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:28:25.269562 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 23405
I20260808 18:28:25.269702 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:28:25.269984 7825 external_mini_cluster.cc:1428] Setting key 18707eeb2c3d507ee2c757484f39ca00
Aug 08 18:28:25 dist-test-slave-jztv krb5kdc[22883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213705, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:28:25.279107 22995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.266558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:48805 (local address 127.7.164.126:44229)
0808 18:28:25.266702 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:25.266706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:25.267544 (+ 838us) server_negotiation.cc:407] Connection header received
0808 18:28:25.268991 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:25.268994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:25.269066 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:25.269176 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:25.270662 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.271361 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.272639 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.272871 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.272983 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.275469 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:25.275473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:25.275476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:28:25.275505 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:25.276871 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:25.277440 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:25.277444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:25.277492 (+ 48us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:25.277608 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:25.277945 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:25.277949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:25.277951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:25.278138 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:25.278265 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:25.278819 (+ 554us) server_negotiation.cc:299] Negotiation successful
0808 18:28:25.278946 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:28:25.279896 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.266946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:44229 (local address 127.7.164.67:48805)
0808 18:28:25.267391 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:25.267434 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:25.268745 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:25.269355 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:25.269362 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:25.269827 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:28:25.270486 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:25.270522 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.271484 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.271487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:25.272494 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:25.272504 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.273021 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.273026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:25.273202 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.273810 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:28:25.273827 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:28:25.275305 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:28:25.276994 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:28:25.277000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:28:25.277014 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:28:25.277268 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:28:25.277691 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:28:25.277695 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:28:25.277697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:28:25.277836 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:28:25.278268 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:28:25.278277 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:28:25.278536 (+ 259us) client_negotiation.cc:769] Sending connection context
0808 18:28:25.278776 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 18:28:25.279105 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":100,"threads_started":1}
I20260808 18:28:25.280943 23595 heartbeater.cc:347] Connected to a master server at 127.7.164.126:44229
I20260808 18:28:25.281148 23595 heartbeater.cc:464] Registering TS with master...
I20260808 18:28:25.281572 23595 heartbeater.cc:511] Master 127.7.164.126:44229 requested a full tablet report, sending...
I20260808 18:28:25.282606 22946 ts_manager.cc:194] Registered new tserver with Master: 7234089f754d47cf954f0a781d7b6105 (127.7.164.67:38299)
I20260808 18:28:25.283355 22946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:48805
I20260808 18:28:25.283849 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:28:25.292979 22995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.285779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44698 (local address 127.7.164.126:44229)
0808 18:28:25.285907 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:25.285910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:25.286150 (+ 240us) server_negotiation.cc:407] Connection header received
0808 18:28:25.286302 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:25.286305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:25.286350 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:25.286421 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:28:25.287222 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.287732 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.288549 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.288760 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.288836 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.290094 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:28:25.290097 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:28:25.290099 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:28:25.290120 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:28:25.291453 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:25.291928 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:25.291931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:25.291932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:25.292004 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:28:25.292235 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:28:25.292237 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:28:25.292238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:28:25.292456 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:28:25.292565 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:25.292767 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 18:28:25.292862 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:28:25.296015 22944 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:28:25.299000 22944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:28:25.314587 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.308471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:45731 (local address 127.0.0.1:34650)
0808 18:28:25.309048 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:25.309064 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:25.309150 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:25.309896 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:25.309898 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:25.309913 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:28:25.310092 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:25.310099 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.311549 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.311555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:25.313177 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:25.313188 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.313991 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.313995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:25.314107 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.314145 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 18:28:25.314300 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 18:28:25.314434 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":208,"thread_start_us":95,"threads_started":1}
I20260808 18:28:25.315294 23617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.308890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34650 (local address 127.7.164.66:45731)
0808 18:28:25.309444 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:25.309454 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:25.309474 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:28:25.309547 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:25.309552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:25.309767 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:25.309936 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:28:25.310261 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.311367 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.313303 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.313858 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.313984 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.314138 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:25.315084 (+ 946us) server_negotiation.cc:299] Negotiation successful
0808 18:28:25.315184 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":112,"threads_started":1}
I20260808 18:28:25.315867 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.307799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51770 (local address 127.7.164.67:38299)
0808 18:28:25.308284 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:25.308300 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:25.308902 (+ 602us) server_negotiation.cc:407] Connection header received
0808 18:28:25.309349 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:25.309354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:25.309532 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:25.309687 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:28:25.310306 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.311422 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.313244 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.314379 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.314498 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.315491 (+ 993us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:25.315550 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 18:28:25.315656 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":115,"threads_started":1}
I20260808 18:28:25.315961 23614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.308202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:42887 (local address 127.0.0.1:34546)
0808 18:28:25.308941 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:25.308957 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:25.309113 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:25.309708 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:25.309714 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:25.309738 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:28:25.309937 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:25.309946 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.311622 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.311625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:25.313043 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:25.313055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.315546 (+ 2491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.315550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:25.315655 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.315682 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:28:25.315739 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 18:28:25.315807 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"mutex_wait_us":128,"thread_start_us":102,"threads_started":1}
I20260808 18:28:25.316072 23612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.307685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:38299 (local address 127.0.0.1:51770)
0808 18:28:25.308655 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:25.308711 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:25.309209 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:25.309867 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:25.309871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:25.309891 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:28:25.310132 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:25.310147 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.311587 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.311593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:25.313129 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:25.313141 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.314842 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.314851 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:25.315006 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.315044 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 18:28:25.315128 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 18:28:25.315229 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":269,"thread_start_us":95,"threads_started":1}
I20260808 18:28:25.316671 23615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.308503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.7.164.65:42887)
0808 18:28:25.308821 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:25.308838 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:25.309074 (+ 236us) server_negotiation.cc:407] Connection header received
0808 18:28:25.309305 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:25.309311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:25.309522 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:25.309688 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:28:25.310083 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.311421 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.313185 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.314124 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.316259 (+ 2135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.316432 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:25.316478 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 18:28:25.316552 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":125,"threads_started":1}
I20260808 18:28:25.318478 23306 tablet_service.cc:1467] Processing CreateTablet for tablet df6e2b036a054192a57bf220bc9891e2 (DEFAULT_TABLE table=test-table [id=b745b72b4913417ca01647ea8d718550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:25.319547 23306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df6e2b036a054192a57bf220bc9891e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:25.319532 23115 tablet_service.cc:1467] Processing CreateTablet for tablet df6e2b036a054192a57bf220bc9891e2 (DEFAULT_TABLE table=test-table [id=b745b72b4913417ca01647ea8d718550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:25.320652 23115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df6e2b036a054192a57bf220bc9891e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:25.320370 23496 tablet_service.cc:1467] Processing CreateTablet for tablet df6e2b036a054192a57bf220bc9891e2 (DEFAULT_TABLE table=test-table [id=b745b72b4913417ca01647ea8d718550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:28:25.321424 23496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df6e2b036a054192a57bf220bc9891e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:28:25.325635 23618 tablet_bootstrap.cc:492] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff: Bootstrap starting.
I20260808 18:28:25.329054 23618 tablet_bootstrap.cc:654] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:25.329089 23624 tablet_bootstrap.cc:492] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105: Bootstrap starting.
I20260808 18:28:25.329775 23622 tablet_bootstrap.cc:492] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030: Bootstrap starting.
I20260808 18:28:25.330538 23618 log.cc:824] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff: Log is configured to *not* fsync() on all Append() calls
I20260808 18:28:25.331275 23624 tablet_bootstrap.cc:654] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:25.331990 23622 tablet_bootstrap.cc:654] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030: Neither blocks nor log segments found. Creating new log.
I20260808 18:28:25.332710 23624 log.cc:824] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105: Log is configured to *not* fsync() on all Append() calls
I20260808 18:28:25.333015 23618 tablet_bootstrap.cc:492] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff: No bootstrap required, opened a new log
I20260808 18:28:25.333576 23618 ts_tablet_manager.cc:1397] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 18:28:25.333570 23622 log.cc:824] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030: Log is configured to *not* fsync() on all Append() calls
I20260808 18:28:25.334849 23624 tablet_bootstrap.cc:492] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105: No bootstrap required, opened a new log
I20260808 18:28:25.335034 23624 ts_tablet_manager.cc:1397] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:28:25.335655 23622 tablet_bootstrap.cc:492] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030: No bootstrap required, opened a new log
I20260808 18:28:25.335867 23622 ts_tablet_manager.cc:1397] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:28:25.337584 23618 raft_consensus.cc:348] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234089f754d47cf954f0a781d7b6105" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38299 } } peers { permanent_uuid: "d60641e02af74d969c10527766a89dff" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45731 } } peers { permanent_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42887 } }
I20260808 18:28:25.337857 23618 raft_consensus.cc:374] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:25.337925 23618 raft_consensus.cc:729] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60641e02af74d969c10527766a89dff, State: Initialized, Role: FOLLOWER
I20260808 18:28:25.338413 23618 consensus_queue.cc:260] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234089f754d47cf954f0a781d7b6105" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38299 } } peers { permanent_uuid: "d60641e02af74d969c10527766a89dff" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45731 } } peers { permanent_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42887 } }
I20260808 18:28:25.338792 23624 raft_consensus.cc:348] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234089f754d47cf954f0a781d7b6105" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38299 } } peers { permanent_uuid: "d60641e02af74d969c10527766a89dff" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45731 } } peers { permanent_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42887 } }
I20260808 18:28:25.339047 23624 raft_consensus.cc:374] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:25.339108 23624 raft_consensus.cc:729] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7234089f754d47cf954f0a781d7b6105, State: Initialized, Role: FOLLOWER
I20260808 18:28:25.339226 23622 raft_consensus.cc:348] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234089f754d47cf954f0a781d7b6105" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38299 } } peers { permanent_uuid: "d60641e02af74d969c10527766a89dff" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45731 } } peers { permanent_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42887 } }
I20260808 18:28:25.339546 23622 raft_consensus.cc:374] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:28:25.339617 23622 raft_consensus.cc:729] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8f1ab3085bd46e2a108e3deb4a28030, State: Initialized, Role: FOLLOWER
I20260808 18:28:25.339560 23624 consensus_queue.cc:260] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234089f754d47cf954f0a781d7b6105" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38299 } } peers { permanent_uuid: "d60641e02af74d969c10527766a89dff" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45731 } } peers { permanent_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42887 } }
I20260808 18:28:25.340080 23618 ts_tablet_manager.cc:1428] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:28:25.340063 23622 consensus_queue.cc:260] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234089f754d47cf954f0a781d7b6105" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38299 } } peers { permanent_uuid: "d60641e02af74d969c10527766a89dff" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45731 } } peers { permanent_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42887 } }
I20260808 18:28:25.340330 23595 heartbeater.cc:503] Master 127.7.164.126:44229 was elected leader, sending a full tablet report...
I20260808 18:28:25.340407 23400 heartbeater.cc:503] Master 127.7.164.126:44229 was elected leader, sending a full tablet report...
I20260808 18:28:25.340497 23624 ts_tablet_manager.cc:1428] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:28:25.341042 23622 ts_tablet_manager.cc:1428] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:28:25.341409 23216 heartbeater.cc:503] Master 127.7.164.126:44229 was elected leader, sending a full tablet report...
I20260808 18:28:25.420225 23637 raft_consensus.cc:482] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:28:25.420436 23637 raft_consensus.cc:504] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234089f754d47cf954f0a781d7b6105" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38299 } } peers { permanent_uuid: "d60641e02af74d969c10527766a89dff" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45731 } } peers { permanent_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42887 } }
I20260808 18:28:25.421527 23637 leader_election.cc:288] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7234089f754d47cf954f0a781d7b6105 (127.7.164.67:38299), d60641e02af74d969c10527766a89dff (127.7.164.66:45731)
I20260808 18:28:25.425575 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.421956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:50949 (local address 127.7.164.67:38299)
0808 18:28:25.422112 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:25.422117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:25.422282 (+ 165us) server_negotiation.cc:407] Connection header received
0808 18:28:25.422494 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:25.422500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:25.422563 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:25.422650 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:28:25.422972 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.423484 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.424563 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.424995 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.425134 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.425192 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:25.425378 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 18:28:25.425445 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:28:25.425590 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.421843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:38299 (local address 127.7.164.65:50949)
0808 18:28:25.422169 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:25.422182 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:25.422354 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:25.422678 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:25.422681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:25.422714 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:28:25.422849 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:25.422855 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.423598 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.423604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:25.424425 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:25.424434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.425136 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.425139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:25.425237 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.425260 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:28:25.425360 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 18:28:25.425440 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":137,"threads_started":1}
I20260808 18:28:25.425839 23617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.422566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:34591 (local address 127.7.164.66:45731)
0808 18:28:25.422692 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:25.422698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:25.422780 (+ 82us) server_negotiation.cc:407] Connection header received
0808 18:28:25.422929 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:25.422932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:25.422985 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:25.423070 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:28:25.423349 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.423838 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.424879 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.425285 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.425432 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.425481 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:25.425668 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 18:28:25.425739 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:28:25.425850 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.422352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:45731 (local address 127.7.164.65:34591)
0808 18:28:25.422678 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:25.422692 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:28:25.422803 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:28:25.423089 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:28:25.423092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:28:25.423108 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:28:25.423237 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:25.423242 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.423948 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.423954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:25.424721 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:28:25.424730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.425423 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.425427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:28:25.425526 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.425555 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 18:28:25.425640 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 18:28:25.425710 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":103,"threads_started":1}
I20260808 18:28:25.426256 23530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df6e2b036a054192a57bf220bc9891e2" candidate_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7234089f754d47cf954f0a781d7b6105" is_pre_election: true
I20260808 18:28:25.426455 23330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df6e2b036a054192a57bf220bc9891e2" candidate_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60641e02af74d969c10527766a89dff" is_pre_election: true
I20260808 18:28:25.426620 23530 raft_consensus.cc:2454] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8f1ab3085bd46e2a108e3deb4a28030 in term 0.
I20260808 18:28:25.426752 23330 raft_consensus.cc:2454] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8f1ab3085bd46e2a108e3deb4a28030 in term 0.
I20260808 18:28:25.427142 23056 leader_election.cc:302] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7234089f754d47cf954f0a781d7b6105, e8f1ab3085bd46e2a108e3deb4a28030; no voters:
I20260808 18:28:25.427402 23637 raft_consensus.cc:2781] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:28:25.427481 23637 raft_consensus.cc:482] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:28:25.427512 23637 raft_consensus.cc:3037] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:25.428730 23637 raft_consensus.cc:504] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234089f754d47cf954f0a781d7b6105" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38299 } } peers { permanent_uuid: "d60641e02af74d969c10527766a89dff" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45731 } } peers { permanent_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42887 } }
I20260808 18:28:25.429069 23637 leader_election.cc:288] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [CANDIDATE]: Term 1 election: Requested vote from peers 7234089f754d47cf954f0a781d7b6105 (127.7.164.67:38299), d60641e02af74d969c10527766a89dff (127.7.164.66:45731)
I20260808 18:28:25.429466 23530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df6e2b036a054192a57bf220bc9891e2" candidate_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7234089f754d47cf954f0a781d7b6105"
I20260808 18:28:25.429587 23330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df6e2b036a054192a57bf220bc9891e2" candidate_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60641e02af74d969c10527766a89dff"
I20260808 18:28:25.429623 23530 raft_consensus.cc:3037] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:25.429752 23330 raft_consensus.cc:3037] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:28:25.430792 23530 raft_consensus.cc:2454] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8f1ab3085bd46e2a108e3deb4a28030 in term 1.
I20260808 18:28:25.430882 23330 raft_consensus.cc:2454] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8f1ab3085bd46e2a108e3deb4a28030 in term 1.
I20260808 18:28:25.431134 23056 leader_election.cc:302] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7234089f754d47cf954f0a781d7b6105, e8f1ab3085bd46e2a108e3deb4a28030; no voters:
I20260808 18:28:25.431331 23637 raft_consensus.cc:2781] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:28:25.431557 23637 raft_consensus.cc:686] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [term 1 LEADER]: Becoming Leader. State: Replica: e8f1ab3085bd46e2a108e3deb4a28030, State: Running, Role: LEADER
I20260808 18:28:25.431893 23637 consensus_queue.cc:237] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234089f754d47cf954f0a781d7b6105" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38299 } } peers { permanent_uuid: "d60641e02af74d969c10527766a89dff" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45731 } } peers { permanent_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42887 } }
I20260808 18:28:25.435348 22943 catalog_manager.cc:5495] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8f1ab3085bd46e2a108e3deb4a28030 (127.7.164.65). New cstate: current_term: 1 leader_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234089f754d47cf954f0a781d7b6105" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60641e02af74d969c10527766a89dff" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8f1ab3085bd46e2a108e3deb4a28030" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42887 } health_report { overall_health: HEALTHY } } }
I20260808 18:28:25.475564 23615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.471950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.7.164.65:42887)
0808 18:28:25.472109 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 18:28:25.472113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:28:25.472127 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:28:25.472215 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:28:25.472218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:28:25.472280 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:28:25.472371 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:28:25.472601 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.473107 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.473958 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:28:25.474173 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:28:25.474337 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:28:25.474786 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:28:25.475223 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:28:25.475320 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 18:28:25.475395 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 18:28:25.475447 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 18:28:25.481590 23330 raft_consensus.cc:1261] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff [term 1 FOLLOWER]: Refusing update from remote peer e8f1ab3085bd46e2a108e3deb4a28030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:28:25.481607 23530 raft_consensus.cc:1261] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105 [term 1 FOLLOWER]: Refusing update from remote peer e8f1ab3085bd46e2a108e3deb4a28030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:28:25.482381 23637 consensus_queue.cc:1035] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60641e02af74d969c10527766a89dff" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:28:25.482623 23640 consensus_queue.cc:1035] T df6e2b036a054192a57bf220bc9891e2 P e8f1ab3085bd46e2a108e3deb4a28030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7234089f754d47cf954f0a781d7b6105" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 38299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:28:25.495385 23646 mvcc.cc:205] Tried to move back new op lower bound from 7316331337645453312 to 7316331337452175360. Current Snapshot: MvccSnapshot[applied={T|T < 7316331337645453312}]
I20260808 18:28:25.500077 22946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:28:25.500240 22946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:28:25.502664 22946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a64de437a6244eb1a1808fceb9517e10: Sending DeleteTablet for 3 replicas of tablet df6e2b036a054192a57bf220bc9891e2
I20260808 18:28:25.503436 23115 tablet_service.cc:1514] Processing DeleteTablet for tablet df6e2b036a054192a57bf220bc9891e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:25 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:34546
I20260808 18:28:25.503614 23306 tablet_service.cc:1514] Processing DeleteTablet for tablet df6e2b036a054192a57bf220bc9891e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:25 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:34650
I20260808 18:28:25.503937 23496 tablet_service.cc:1514] Processing DeleteTablet for tablet df6e2b036a054192a57bf220bc9891e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:28:25 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:51770
I20260808 18:28:25.504093 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 23030
I20260808 18:28:25.512987 23663 tablet_replica.cc:330] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105: stopping tablet replica
I20260808 18:28:25.513278 23663 raft_consensus.cc:2229] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:28:25.513671 23663 raft_consensus.cc:2258] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:28:25.515082 23663 ts_tablet_manager.cc:1905] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:28:25.517292 23663 ts_tablet_manager.cc:1918] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:28:25.517417 23663 log.cc:1196] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/df6e2b036a054192a57bf220bc9891e2
I20260808 18:28:25.517648 23660 tablet_replica.cc:330] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff: stopping tablet replica
I20260808 18:28:25.517807 23663 ts_tablet_manager.cc:1939] T df6e2b036a054192a57bf220bc9891e2 P 7234089f754d47cf954f0a781d7b6105: Deleting consensus metadata
I20260808 18:28:25.518006 23660 raft_consensus.cc:2229] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:28:25.518378 23660 raft_consensus.cc:2258] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:28:25.519237 22928 catalog_manager.cc:4841] TS 7234089f754d47cf954f0a781d7b6105 (127.7.164.67:38299): tablet df6e2b036a054192a57bf220bc9891e2 (table test-table [id=b745b72b4913417ca01647ea8d718550]) successfully deleted
I20260808 18:28:25.519994 23660 ts_tablet_manager.cc:1905] T df6e2b036a054192a57bf220bc9891e2 P d60641e02af74d969c10527766a89dff: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 18:28:25.520776 22925 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:42887 (error 108)
I20260808 18:28:25.520812 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 23221
I20260808 18:28:25.527285 23612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.526985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:42887 (local address 127.0.0.1:34578)
0808 18:28:25.527128 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:25.527205 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:42887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 18:28:25.527458 22925 catalog_manager.cc:4584] TS e8f1ab3085bd46e2a108e3deb4a28030 (127.7.164.65:42887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df6e2b036a054192a57bf220bc9891e2: Network error: Client connection negotiation failed: client connection to 127.7.164.65:42887: connect: Connection refused (error 111)
I20260808 18:28:25.531764 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 23405
I20260808 18:28:25.538647 23612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:28:25.531897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:45731 (local address 127.0.0.1:34664)
0808 18:28:25.538405 (+ 6508us) negotiation.cc:107] Waiting for socket to connect
0808 18:28:25.538503 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.66:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6370}
I20260808 18:28:25.545209 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 22901
2026-08-08T18:28:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2605 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:28:25 dist-test-slave-jztv krb5kdc[23667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:28:25 dist-test-slave-jztv krb5kdc[23667](info): set up 2 sockets
Aug 08 18:28:25 dist-test-slave-jztv krb5kdc[23667](info): commencing operation
krb5kdc: starting...
W20260808 18:28:26.779336 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:28:26 dist-test-slave-jztv krb5kdc[23667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:28:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:28:26Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.164.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.164.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/rangeradmin_127.7.164.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/rangeradmin_127.7.164.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.164.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.164.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/rangerlookup_127.7.164.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/rangerlookup_127.7.164.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.164.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.164.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/HTTP_127.7.164.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/HTTP_127.7.164.84@KRBTEST.COM.keytab.
I20260808 18:28:26.963575 7825 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasklhTizW/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/postgres -l logfile start
2026-08-08 18:28:29.511 UTC [23702] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 18:28:29.511 UTC [23702] LOG: listening on IPv4 address "127.7.164.84", port 41053
2026-08-08 18:28:29.515 UTC [23702] LOG: listening on Unix socket "/tmp/.s.PGSQL.41053"
2026-08-08 18:28:29.552 UTC [23707] LOG: database system was shut down at 2026-08-08 18:28:28 UTC
2026-08-08 18:28:29.556 UTC [23702] LOG: database system is ready to accept connections
I20260808 18:28:30.696382 7825 mini_postgres.cc:96] Postgres bound to 41053
2026-08-08 18:28:30.704 UTC [23719] FATAL: database "slave" does not exist
127.7.164.84:41053 - accepting connections
I20260808 18:28:30.704986 7825 mini_ranger.cc:161] Starting Ranger...
I20260808 18:28:30.721033 7825 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 18:28:31.209589 7825 mini_ranger.cc:87] Created ranger Postgres database
I20260808 18:28:31.209694 7825 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-admin
2026-08-08 18:28:31,490 [I] DB FLAVOR :POSTGRES
2026-08-08 18:28:31,491 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 18:28:31,491 [I] Checking connection..
2026-08-08 18:28:31,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 18:28:31,750 [I] Checking connection passed.
2026-08-08 18:28:31,750 [I] --------- Verifying version history table ---------
2026-08-08 18:28:31,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:28:32,020 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 18:28:32,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:28:32,284 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 18:28:32,284 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 18:28:32,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 18:28:32.563 UTC [23812] WARNING: there is no transaction in progress
2026-08-08 18:28:32,570 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 18:28:32,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:28:32,841 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 18:28:32,841 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 18:28:32,841 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 18:28:33,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 18:28:33,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-jztv.c.gcp-upstream.internal','N') ;"
2026-08-08 18:28:33,629 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 18:28:33,885 [I] Table x_portal_user does not exist in database ranger
2026-08-08 18:28:33,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 18:28:34,140 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 18:28:34,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 18:28:34,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 18:28:34,652 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 18:28:34,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 18:28:35.644 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.660 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.675 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.690 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.706 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.724 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.778 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.785 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.799 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.809 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.819 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.833 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.843 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.851 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.858 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:35.865 UTC [23969] WARNING: there is no transaction in progress
2026-08-08 18:28:36,517 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 18:28:36,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 18:28:36,776 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 18:28:36,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 18:28:37,022 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 18:28:37,022 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 18:28:37,287 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 18:28:37,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 18:28:37,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 18:28:37,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-jztv.c.gcp-upstream.internal';"
2026-08-08 18:28:38,057 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 18:28:38,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 18:28:38,322 [I] DB_PATCHES have already been applied
I20260808 18:28:38.328043 7825 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-admin:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasklhTizW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 18:28:38.328122 7825 mini_ranger.cc:193] Using host: 127.7.164.84
I20260808 18:28:38.332659 7825 mini_ranger.cc:227] Ranger admin URL: http://127.7.164.84:56987
Aug 08, 2026 6:28:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:28:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:28:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 6:28:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 6:28:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 6:28:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 6:28:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 18:28:39.653106 7825 mini_ranger.cc:160] Time spent starting Ranger: real 8.948s user 0.003s sys 0.010s
Aug 08, 2026 6:28:59 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 6:28:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 6:28:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 6:28:59 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 6:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 6:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 6:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 6:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 6:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 6:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 6:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 6:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 6:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 6:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 6:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 6:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 6:28:59 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 6:29:00 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 18:29:01.225950 7825 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.164.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.164.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/rangerkms_127.7.164.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/rangerkms_127.7.164.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.164.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.164.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.164.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/HTTP_127.7.164.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.164.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/HTTP_127.7.164.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 18:29:01 dist-test-slave-jztv krb5kdc[23667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213741, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 18:29:01.313964 7825 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 18:29:01.324817 7825 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 18:29:02.183581 7825 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 18:29:02.183686 7825 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-kms
I20260808 18:29:02.183789 7825 mini_ranger_kms.cc:227] Using postgres at 127.7.164.84:41053
2026-08-08 18:29:03,096 [I] DB FLAVOR :POSTGRES
2026-08-08 18:29:03,096 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 18:29:03,097 [I] Checking connection
2026-08-08 18:29:03,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 18:29:03,436 [I] connection success
2026-08-08 18:29:03,437 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 18:29:03,437 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 18:29:03,437 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 18:29:03,718 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 18:29:03,718 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 18:29:03,718 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 18:29:03,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:41053/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 18:29:04,000 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 18:29:04.470652 7825 mini_ranger_kms.cc:310] Created kms service
I20260808 18:29:04.718010 7825 mini_ranger_kms.cc:326] Created kudu user
I20260808 18:29:04.762122 7825 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 18:29:04 dist-test-slave-jztv krb5kdc[23667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213741, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:29:05.154197 7825 mini_ranger_kms.cc:384] Added ranger policy
I20260808 18:29:05.155114 7825 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-kms:/tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 18:29:05.155155 7825 mini_ranger_kms.cc:240] Using host: 127.7.164.84
I20260808 18:29:05.160270 7825 mini_ranger_kms.cc:276] Ranger KMS PID: 24207
I20260808 18:29:05.160418 7825 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.164.84:50075
Aug 08, 2026 6:29:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:29:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:29:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 6:29:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 6:29:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 6:29:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 18:29:08.117391 7825 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.803s user 0.008s sys 0.627s
I20260808 18:29:08.117579 7825 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 18:29:08.117719 7825 mini_ranger_kms.cc:401] 127.7.164.84:50075/kms/v1/keys
Aug 08 18:29:08 dist-test-slave-jztv krb5kdc[23667](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213748, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.164.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 18:29:08 dist-test-slave-jztv krb5kdc[23667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213748, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.164.84@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
Aug 08 18:29:09 dist-test-slave-jztv krb5kdc[23667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213748, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.164.84@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
Aug 08 18:29:11 dist-test-slave-jztv krb5kdc[23667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:29:11.817493 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:34305
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:34305
--ranger_config_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.164.84:50075/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:29:11.987396 24272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:29:11.987681 24272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:29:11.987723 24272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:29:11.991927 24272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:29:11.991986 24272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:29:11.992005 24272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:29:11.992022 24272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:29:11.992038 24272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:29:11.997984 24272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.164.84:50075/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:34305
--ranger_config_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:34305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:29:11.999228 24272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:29:12.000355 24272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:29:12.012601 24282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:29:12.012657 24284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:29:12.012810 24272 server_base.cc:1034] running on GCE node
W20260808 18:29:12.012836 24281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:29:12.013604 24272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:29:12.014828 24272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:29:12.016163 24272 hybrid_clock.cc:648] HybridClock initialized: now 1786213752016146 us; error 36 us; skew 500 ppm
Aug 08 18:29:12 dist-test-slave-jztv krb5kdc[23667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213752, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:29:12.021904 24272 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:29:12.022933 24272 webserver.cc:460] Webserver started at http://127.7.164.126:38899/ using document root <none> and password file <none>
I20260808 18:29:12.023511 24272 fs_manager.cc:359] Metadata directory not provided
I20260808 18:29:12.023568 24272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:29:12.023797 24272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:29:12 dist-test-slave-jztv krb5kdc[23667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213752, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:29:12.088068 24272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "f950bdfaafda411ea6d69fabcf2e715a"
format_stamp: "Formatted at 2026-08-08 18:29:12 on dist-test-slave-jztv"
server_key: "1c73e422d0d48009f00ee0e81721a25f"
server_key_iv: "e3fb6910b0cd421bd527ece1bfa65663"
server_key_version: "kuduclusterkey@0"
I20260808 18:29:12.088768 24272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "f950bdfaafda411ea6d69fabcf2e715a"
format_stamp: "Formatted at 2026-08-08 18:29:12 on dist-test-slave-jztv"
server_key: "1c73e422d0d48009f00ee0e81721a25f"
server_key_iv: "e3fb6910b0cd421bd527ece1bfa65663"
server_key_version: "kuduclusterkey@0"
I20260808 18:29:12.093005 24272 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:29:12.111240 24295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:29:12.112681 24272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:29:12.112829 24272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "f950bdfaafda411ea6d69fabcf2e715a"
format_stamp: "Formatted at 2026-08-08 18:29:12 on dist-test-slave-jztv"
server_key: "1c73e422d0d48009f00ee0e81721a25f"
server_key_iv: "e3fb6910b0cd421bd527ece1bfa65663"
server_key_version: "kuduclusterkey@0"
I20260808 18:29:12.112967 24272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:29:12.127300 24272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:29:12.130263 24272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:29:12.130479 24272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:29:12.145892 24272 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:34305
I20260808 18:29:12.145975 24368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:34305 every 8 connection(s)
I20260808 18:29:12.147102 24272 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:29:12.148703 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 24272
I20260808 18:29:12.148839 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:29:12.149159 7825 external_mini_cluster.cc:1428] Setting key 3659ce08fafeaa23da24cac23d0b8875
I20260808 18:29:12.150818 24369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:29:12.159682 24369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a: Bootstrap starting.
Aug 08 18:29:12 dist-test-slave-jztv krb5kdc[23667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:29:12.163441 24369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a: Neither blocks nor log segments found. Creating new log.
I20260808 18:29:12.165293 24369 log.cc:824] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:29:12.167992 24369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a: No bootstrap required, opened a new log
I20260808 18:29:12.169200 24375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:12.151190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56934 (local address 127.7.164.126:34305)
0808 18:29:12.151927 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0808 18:29:12.151987 (+ 60us) server_negotiation.cc:399] Waiting for connection header
0808 18:29:12.152036 (+ 49us) server_negotiation.cc:407] Connection header received
0808 18:29:12.153465 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:29:12.153492 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:29:12.153935 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:29:12.154385 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:29:12.155584 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:12.157582 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:12.158961 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:12.159721 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:12.160354 (+ 633us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:12.163155 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:29:12.163162 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:29:12.163181 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:29:12.163216 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:29:12.166148 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:29:12.166713 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:29:12.166718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:29:12.166723 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:29:12.166833 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:29:12.167159 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:29:12.167163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:29:12.167164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:29:12.167564 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:29:12.167775 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 18:29:12.168091 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 18:29:12.168408 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":312,"threads_started":1}
I20260808 18:29:12.172741 24369 raft_consensus.cc:348] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f950bdfaafda411ea6d69fabcf2e715a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34305 } }
I20260808 18:29:12.173029 24369 raft_consensus.cc:374] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:29:12.173128 24369 raft_consensus.cc:729] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f950bdfaafda411ea6d69fabcf2e715a, State: Initialized, Role: FOLLOWER
I20260808 18:29:12.173621 24369 consensus_queue.cc:260] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f950bdfaafda411ea6d69fabcf2e715a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34305 } }
I20260808 18:29:12.173784 24369 raft_consensus.cc:388] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:29:12.173849 24369 raft_consensus.cc:482] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:29:12.173950 24369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:29:12.175563 24369 raft_consensus.cc:504] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f950bdfaafda411ea6d69fabcf2e715a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34305 } }
I20260808 18:29:12.175958 24369 leader_election.cc:302] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f950bdfaafda411ea6d69fabcf2e715a; no voters:
I20260808 18:29:12.176421 24369 leader_election.cc:288] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:29:12.176594 24377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:29:12.176960 24377 raft_consensus.cc:686] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [term 1 LEADER]: Becoming Leader. State: Replica: f950bdfaafda411ea6d69fabcf2e715a, State: Running, Role: LEADER
I20260808 18:29:12.177351 24377 consensus_queue.cc:237] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f950bdfaafda411ea6d69fabcf2e715a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34305 } }
I20260808 18:29:12.177874 24369 sys_catalog.cc:564] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:29:12.179543 24378 sys_catalog.cc:455] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f950bdfaafda411ea6d69fabcf2e715a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f950bdfaafda411ea6d69fabcf2e715a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34305 } } }
I20260808 18:29:12.179762 24378 sys_catalog.cc:458] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [sys.catalog]: This master's current role is: LEADER
I20260808 18:29:12.180150 24378 sys_catalog.cc:455] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f950bdfaafda411ea6d69fabcf2e715a. Latest consensus state: current_term: 1 leader_uuid: "f950bdfaafda411ea6d69fabcf2e715a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f950bdfaafda411ea6d69fabcf2e715a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 34305 } } }
I20260808 18:29:12.180291 24378 sys_catalog.cc:458] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a [sys.catalog]: This master's current role is: LEADER
I20260808 18:29:12.181226 24391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:29:12.184670 24391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:29:12.185300 24369 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 18:29:12.199047 24391 catalog_manager.cc:1347] Generated new cluster ID: 8f6aca5ed47347b4bd98821d2d994350
I20260808 18:29:12.199344 24391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:29:12.218035 24391 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:29:12.219805 24391 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:29:12.229162 24391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a: Generated new TSK 0
I20260808 18:29:12.230053 24391 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:29:13.145641 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.164.84:50075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:34305
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:29:13.376304 24433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:29:13.376678 24433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:29:13.376765 24433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:29:13.381314 24433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:29:13.381394 24433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:29:13.381494 24433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:29:13.388502 24433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.164.84:50075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:34305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:29:13.390075 24433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:29:13.391253 24433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:29:13.406045 24440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:29:13.406299 24439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:29:13.406153 24444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:29:13.406771 24433 server_base.cc:1034] running on GCE node
I20260808 18:29:13.407267 24433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:29:13.408087 24433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:29:13.409298 24433 hybrid_clock.cc:648] HybridClock initialized: now 1786213753409307 us; error 46 us; skew 500 ppm
Aug 08 18:29:13 dist-test-slave-jztv krb5kdc[23667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213753, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:29:13.413316 24433 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:29:13.414613 24433 webserver.cc:460] Webserver started at http://127.7.164.65:37883/ using document root <none> and password file <none>
I20260808 18:29:13.415356 24433 fs_manager.cc:359] Metadata directory not provided
I20260808 18:29:13.415505 24433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:29:13.415802 24433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:29:13 dist-test-slave-jztv krb5kdc[23667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213753, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:29:13.433885 24433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "d2aa5a4e5a5a41b9aebfded4c943521b"
format_stamp: "Formatted at 2026-08-08 18:29:13 on dist-test-slave-jztv"
server_key: "f0a105d000ec532f3049f38a96dcb68d"
server_key_iv: "baafc903bd80e1f54354a34656dc3414"
server_key_version: "kuduclusterkey@0"
I20260808 18:29:13.434782 24433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "d2aa5a4e5a5a41b9aebfded4c943521b"
format_stamp: "Formatted at 2026-08-08 18:29:13 on dist-test-slave-jztv"
server_key: "f0a105d000ec532f3049f38a96dcb68d"
server_key_iv: "baafc903bd80e1f54354a34656dc3414"
server_key_version: "kuduclusterkey@0"
I20260808 18:29:13.442796 24433 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.000s
I20260808 18:29:13.458685 24463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:29:13.460153 24433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:29:13.460297 24433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "d2aa5a4e5a5a41b9aebfded4c943521b"
format_stamp: "Formatted at 2026-08-08 18:29:13 on dist-test-slave-jztv"
server_key: "f0a105d000ec532f3049f38a96dcb68d"
server_key_iv: "baafc903bd80e1f54354a34656dc3414"
server_key_version: "kuduclusterkey@0"
I20260808 18:29:13.460455 24433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:29:13.484727 24433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:29:13.488336 24433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:29:13.488561 24433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:29:13.489363 24433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:29:13.490435 24433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:29:13.490549 24433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:29:13.490626 24433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:29:13.490680 24433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:29:13.511572 24433 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:39829
I20260808 18:29:13.511806 24629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:39829 every 8 connection(s)
I20260808 18:29:13.512848 24433 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:29:13.514165 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 24433
I20260808 18:29:13.514322 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:29:13.514667 7825 external_mini_cluster.cc:1428] Setting key da8b2ffa2ac679051a63d9a0bcf69ca7
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
Aug 08 18:29:13 dist-test-slave-jztv krb5kdc[23667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213753, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:29:13.558369 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:13.533520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:36855 (local address 127.7.164.126:34305)
0808 18:29:13.541664 (+ 8144us) server_negotiation.cc:206] Beginning negotiation
0808 18:29:13.541671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:29:13.541693 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:29:13.541747 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:29:13.541751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:29:13.541814 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:29:13.541977 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:29:13.543698 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:13.544667 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:13.546305 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:13.546855 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:13.547046 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:13.552921 (+ 5875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:29:13.552927 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:29:13.552929 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:29:13.552965 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:29:13.555293 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:29:13.556108 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:29:13.556115 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:29:13.556118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:29:13.556226 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:29:13.556722 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:29:13.556725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:29:13.556727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:29:13.557031 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:29:13.557126 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:29:13.557981 (+ 855us) server_negotiation.cc:299] Negotiation successful
0808 18:29:13.558143 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8012,"thread_start_us":108,"threads_started":1}
I20260808 18:29:13.560415 24634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:13.534029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:34305 (local address 127.7.164.65:36855)
0808 18:29:13.535738 (+ 1709us) negotiation.cc:107] Waiting for socket to connect
0808 18:29:13.535803 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 18:29:13.537400 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:29:13.542234 (+ 4834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:29:13.542248 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:29:13.542633 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:29:13.543426 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:13.543464 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:13.544916 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:13.544919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:13.546134 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:13.546145 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:13.547288 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:13.547294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:13.547675 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:13.550439 (+ 2764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:29:13.550469 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:29:13.552604 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:29:13.555481 (+ 2877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:29:13.555489 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:29:13.555514 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:29:13.555955 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:29:13.556359 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:29:13.556361 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:29:13.556365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:29:13.556593 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:29:13.557212 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:29:13.557222 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:29:13.557624 (+ 402us) client_negotiation.cc:769] Sending connection context
0808 18:29:13.558877 (+ 1253us) client_negotiation.cc:240] Negotiation successful
0808 18:29:13.559271 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1433,"thread_start_us":272,"threads_started":1}
I20260808 18:29:13.561928 24630 heartbeater.cc:347] Connected to a master server at 127.7.164.126:34305
I20260808 18:29:13.562180 24630 heartbeater.cc:464] Registering TS with master...
I20260808 18:29:13.562825 24630 heartbeater.cc:511] Master 127.7.164.126:34305 requested a full tablet report, sending...
I20260808 18:29:13.564661 24317 ts_manager.cc:194] Registered new tserver with Master: d2aa5a4e5a5a41b9aebfded4c943521b (127.7.164.65:39829)
I20260808 18:29:13.566257 24317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:36855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:29:13.569940 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.164.84:50075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:34305
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:29:13 dist-test-slave-jztv krb5kdc[23667](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213753, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 18:29:13.805294 24641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:29:13.805725 24641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:29:13.805810 24641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:29:13.812808 24641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:29:13.812906 24641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:29:13.813025 24641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:29:13.820950 24641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.164.84:50075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:34305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:29:13.822103 24641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:29:13.823220 24641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:29:13.834473 24654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:29:13.834424 24650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:29:13.834363 24651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:29:13.834695 24641 server_base.cc:1034] running on GCE node
I20260808 18:29:13.835216 24641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:29:13.835948 24641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:29:13.837145 24641 hybrid_clock.cc:648] HybridClock initialized: now 1786213753837113 us; error 48 us; skew 500 ppm
Aug 08 18:29:13 dist-test-slave-jztv krb5kdc[23667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213753, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:29:13.840770 24641 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:29:13.841892 24641 webserver.cc:460] Webserver started at http://127.7.164.66:39199/ using document root <none> and password file <none>
I20260808 18:29:13.842628 24641 fs_manager.cc:359] Metadata directory not provided
I20260808 18:29:13.842725 24641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:29:13.843108 24641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:29:13 dist-test-slave-jztv krb5kdc[23667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213753, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:29:13.859261 24641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "7fea9b08587840e2b031b06e1cf90afb"
format_stamp: "Formatted at 2026-08-08 18:29:13 on dist-test-slave-jztv"
server_key: "e18f393246daf29a9dd72e3c5b07cd56"
server_key_iv: "bda4eeb1b888ef45e996c087d1aa8b2e"
server_key_version: "kuduclusterkey@0"
I20260808 18:29:13.860196 24641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "7fea9b08587840e2b031b06e1cf90afb"
format_stamp: "Formatted at 2026-08-08 18:29:13 on dist-test-slave-jztv"
server_key: "e18f393246daf29a9dd72e3c5b07cd56"
server_key_iv: "bda4eeb1b888ef45e996c087d1aa8b2e"
server_key_version: "kuduclusterkey@0"
I20260808 18:29:13.866106 24641 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260808 18:29:13.887117 24663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:29:13.889287 24641 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:29:13.889545 24641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "7fea9b08587840e2b031b06e1cf90afb"
format_stamp: "Formatted at 2026-08-08 18:29:13 on dist-test-slave-jztv"
server_key: "e18f393246daf29a9dd72e3c5b07cd56"
server_key_iv: "bda4eeb1b888ef45e996c087d1aa8b2e"
server_key_version: "kuduclusterkey@0"
I20260808 18:29:13.889806 24641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:29:13.913489 24641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:29:13.918109 24641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:29:13.918569 24641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:29:13.919636 24641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:29:13.921176 24641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:29:13.921249 24641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:29:13.921310 24641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:29:13.921379 24641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:29:13.943132 24641 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:43923
I20260808 18:29:13.943462 24821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:43923 every 8 connection(s)
I20260808 18:29:13.944270 24641 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:29:13.954320 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 24641
I20260808 18:29:13.954488 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:29:13.954782 7825 external_mini_cluster.cc:1428] Setting key cba513186cf0d8b0b7fd0416712de77c
Aug 08 18:29:13 dist-test-slave-jztv krb5kdc[23667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213753, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:29:13.967073 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:13.946943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:41217 (local address 127.7.164.126:34305)
0808 18:29:13.947113 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 18:29:13.947118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:29:13.948224 (+ 1106us) server_negotiation.cc:407] Connection header received
0808 18:29:13.950078 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:29:13.950081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:29:13.950145 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:29:13.950279 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:29:13.952103 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:13.952938 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:13.955002 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:13.955318 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:13.955448 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:13.961269 (+ 5821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:29:13.961274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:29:13.961278 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:29:13.961311 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:29:13.963461 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:29:13.964281 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:29:13.964286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:29:13.964288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:29:13.964421 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:29:13.965000 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:29:13.965004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:29:13.965007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:29:13.965410 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:29:13.965558 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 18:29:13.966690 (+ 1132us) server_negotiation.cc:299] Negotiation successful
0808 18:29:13.966872 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:29:13.968605 24824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:13.947333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:34305 (local address 127.7.164.66:41217)
0808 18:29:13.947988 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0808 18:29:13.948071 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0808 18:29:13.949788 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:29:13.950475 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:29:13.950486 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:29:13.950970 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:29:13.951855 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:13.951890 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:13.953123 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:13.953128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:13.954858 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:13.954871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:13.955514 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:13.955518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:13.955834 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:13.959341 (+ 3507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:29:13.959368 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:29:13.961003 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:29:13.963636 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:29:13.963647 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:29:13.963671 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:29:13.964116 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:29:13.964558 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:29:13.964563 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:29:13.964568 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:29:13.964839 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:29:13.965568 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:29:13.965576 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:29:13.966133 (+ 557us) client_negotiation.cc:769] Sending connection context
0808 18:29:13.966567 (+ 434us) client_negotiation.cc:240] Negotiation successful
0808 18:29:13.967053 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":146,"threads_started":1}
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
I20260808 18:29:13.970435 24822 heartbeater.cc:347] Connected to a master server at 127.7.164.126:34305
I20260808 18:29:13.970762 24822 heartbeater.cc:464] Registering TS with master...
I20260808 18:29:13.971292 24822 heartbeater.cc:511] Master 127.7.164.126:34305 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
I20260808 18:29:13.972478 24317 ts_manager.cc:194] Registered new tserver with Master: 7fea9b08587840e2b031b06e1cf90afb (127.7.164.66:43923)
I20260808 18:29:13.973285 24317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:41217
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:29:13.989252 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.164.84:50075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:34305
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:29:14.204852 24829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:29:14.205308 24829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:29:14.205411 24829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:29:14.210487 24829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:29:14.210566 24829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:29:14.210671 24829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:29:14.218029 24829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.164.84:50075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:34305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:29:14.219493 24829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:29:14.220707 24829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:29:14.235992 24838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:29:14.236337 24841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:29:14.236609 24836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:29:14.237812 24829 server_base.cc:1034] running on GCE node
I20260808 18:29:14.238722 24829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:29:14.239516 24829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:29:14.240697 24829 hybrid_clock.cc:648] HybridClock initialized: now 1786213754240662 us; error 61 us; skew 500 ppm
Aug 08 18:29:14 dist-test-slave-jztv krb5kdc[23667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213754, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:29:14.244472 24829 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:29:14.245553 24829 webserver.cc:460] Webserver started at http://127.7.164.67:38873/ using document root <none> and password file <none>
I20260808 18:29:14.246343 24829 fs_manager.cc:359] Metadata directory not provided
I20260808 18:29:14.246488 24829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:29:14.246865 24829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:29:14 dist-test-slave-jztv krb5kdc[23667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213754, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:29:14.263509 24829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "667689b615854325b03a2f876f887ff2"
format_stamp: "Formatted at 2026-08-08 18:29:14 on dist-test-slave-jztv"
server_key: "cf3e907ae988a1de886a7ea14842ee6b"
server_key_iv: "fbfd61088e631bc7caf95c3aab1a74e4"
server_key_version: "kuduclusterkey@0"
I20260808 18:29:14.264683 24829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "667689b615854325b03a2f876f887ff2"
format_stamp: "Formatted at 2026-08-08 18:29:14 on dist-test-slave-jztv"
server_key: "cf3e907ae988a1de886a7ea14842ee6b"
server_key_iv: "fbfd61088e631bc7caf95c3aab1a74e4"
server_key_version: "kuduclusterkey@0"
I20260808 18:29:14.274307 24829 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.003s sys 0.002s
I20260808 18:29:14.294961 24852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:29:14.297578 24829 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.001s
I20260808 18:29:14.297884 24829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "667689b615854325b03a2f876f887ff2"
format_stamp: "Formatted at 2026-08-08 18:29:14 on dist-test-slave-jztv"
server_key: "cf3e907ae988a1de886a7ea14842ee6b"
server_key_iv: "fbfd61088e631bc7caf95c3aab1a74e4"
server_key_version: "kuduclusterkey@0"
I20260808 18:29:14.298152 24829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:29:14.316970 24829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:29:14.320948 24829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:29:14.321416 24829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:29:14.322511 24829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:29:14.324263 24829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:29:14.324620 24829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:29:14.324728 24829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:29:14.324796 24829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:29:14 dist-test-slave-jztv krb5kdc[23667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213753, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:29:14.364436 24829 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:34547
I20260808 18:29:14.365831 24829 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:29:14.367673 25004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:34547 every 8 connection(s)
I20260808 18:29:14.374526 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 24829
I20260808 18:29:14.374662 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:29:14.374985 7825 external_mini_cluster.cc:1428] Setting key e514ba50c3a28bf4a240548b6268c441
Aug 08 18:29:14 dist-test-slave-jztv krb5kdc[23667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213754, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:29:14.388197 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.372287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:47053 (local address 127.7.164.126:34305)
0808 18:29:14.372412 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:29:14.372417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:29:14.373240 (+ 823us) server_negotiation.cc:407] Connection header received
0808 18:29:14.374958 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:29:14.374962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:29:14.375030 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:29:14.375079 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:29:14.377133 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.377943 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.379916 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.380264 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.380335 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:14.383820 (+ 3485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:29:14.383825 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:29:14.383829 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:29:14.383882 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:29:14.385680 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:29:14.386343 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:29:14.386347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:29:14.386349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:29:14.386447 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:29:14.386878 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:29:14.386882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:29:14.386884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:29:14.387118 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:29:14.387222 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 18:29:14.387850 (+ 628us) server_negotiation.cc:299] Negotiation successful
0808 18:29:14.388005 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:29:14.389925 25009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.371077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:34305 (local address 127.7.164.67:47053)
0808 18:29:14.373086 (+ 2009us) negotiation.cc:107] Waiting for socket to connect
0808 18:29:14.373143 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 18:29:14.374662 (+ 1519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:29:14.375543 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:29:14.375554 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:29:14.376139 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:29:14.376954 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:14.376976 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.378121 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.378124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:14.379786 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:14.379798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.380544 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.380548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:14.380805 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:14.381560 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:29:14.381580 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:29:14.383611 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:29:14.385809 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:29:14.385816 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:29:14.385834 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:29:14.386207 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:29:14.386560 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:29:14.386563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:29:14.386565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:29:14.386746 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:29:14.387230 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:29:14.387237 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:29:14.387552 (+ 315us) client_negotiation.cc:769] Sending connection context
0808 18:29:14.388415 (+ 863us) client_negotiation.cc:240] Negotiation successful
0808 18:29:14.388761 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1753,"thread_start_us":216,"threads_started":1}
I20260808 18:29:14.391250 25006 heartbeater.cc:347] Connected to a master server at 127.7.164.126:34305
I20260808 18:29:14.391568 25006 heartbeater.cc:464] Registering TS with master...
I20260808 18:29:14.392258 25006 heartbeater.cc:511] Master 127.7.164.126:34305 requested a full tablet report, sending...
I20260808 18:29:14.393590 24317 ts_manager.cc:194] Registered new tserver with Master: 667689b615854325b03a2f876f887ff2 (127.7.164.67:34547)
I20260808 18:29:14.394567 24317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:47053
I20260808 18:29:14.401943 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:29:14.412592 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.404548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56936 (local address 127.7.164.126:34305)
0808 18:29:14.404670 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:29:14.404674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:29:14.404689 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:29:14.404735 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:29:14.404738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:29:14.404781 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:29:14.404877 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:29:14.405762 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.406390 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.407278 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.407510 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.407620 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:14.409032 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:29:14.409036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:29:14.409039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:29:14.409065 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:29:14.410793 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:29:14.411221 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:29:14.411226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:29:14.411228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:29:14.411316 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:29:14.411606 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:29:14.411611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:29:14.411612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:29:14.411798 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:29:14.411939 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 18:29:14.412189 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 18:29:14.412308 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:29:14.415513 24317 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:29:14.419078 24317 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:29:14.438319 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.429266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55900 (local address 127.7.164.65:39829)
0808 18:29:14.431806 (+ 2540us) server_negotiation.cc:206] Beginning negotiation
0808 18:29:14.431825 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:29:14.431855 (+ 30us) server_negotiation.cc:407] Connection header received
0808 18:29:14.431937 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:29:14.431943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:29:14.432140 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:29:14.432308 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:29:14.432616 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.433789 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.435577 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.436477 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.436583 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:14.436729 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 18:29:14.437957 (+ 1228us) server_negotiation.cc:299] Negotiation successful
0808 18:29:14.438095 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2412,"thread_start_us":152,"threads_started":1}
I20260808 18:29:14.437352 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.428720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:39829 (local address 127.0.0.1:55900)
0808 18:29:14.430427 (+ 1707us) negotiation.cc:107] Waiting for socket to connect
0808 18:29:14.430473 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 18:29:14.430598 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:29:14.432296 (+ 1698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:29:14.432302 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:29:14.432325 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:29:14.432469 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:14.432478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.433938 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.433941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:14.435434 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:14.435446 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.436787 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.436790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:14.436905 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:14.436945 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 18:29:14.437062 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 18:29:14.437200 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1475,"thread_start_us":2137,"threads_started":1}
I20260808 18:29:14.439954 25027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.428814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:43923 (local address 127.0.0.1:37406)
0808 18:29:14.432586 (+ 3772us) negotiation.cc:107] Waiting for socket to connect
0808 18:29:14.432602 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:29:14.432784 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:29:14.433437 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:29:14.433440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:29:14.433457 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:29:14.433539 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:14.433544 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.435837 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.435840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:14.437709 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:14.437719 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.439578 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.439582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:14.439671 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:14.439692 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:29:14.439756 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 18:29:14.439818 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3702,"thread_start_us":1246,"threads_started":1}
I20260808 18:29:14.440469 25025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.429685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37406 (local address 127.7.164.66:43923)
0808 18:29:14.430021 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0808 18:29:14.430077 (+ 56us) server_negotiation.cc:399] Waiting for connection header
0808 18:29:14.432709 (+ 2632us) server_negotiation.cc:407] Connection header received
0808 18:29:14.432934 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:29:14.432940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:29:14.433125 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:29:14.433258 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:29:14.434540 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.435704 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.437993 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.439192 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.439278 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:14.439472 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 18:29:14.440186 (+ 714us) server_negotiation.cc:299] Negotiation successful
0808 18:29:14.440293 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":142,"threads_started":1}
I20260808 18:29:14.443413 24735 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa1880d7bef4a839ecf740b7cce003c (DEFAULT_TABLE table=test-table [id=32e263aac739480096cd99e4dfb55e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:29:14.444700 24735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa1880d7bef4a839ecf740b7cce003c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:29:14.449376 25032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.431146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:34547 (local address 127.0.0.1:60762)
0808 18:29:14.435965 (+ 4819us) negotiation.cc:107] Waiting for socket to connect
0808 18:29:14.435982 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:29:14.436086 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:29:14.443240 (+ 7154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:29:14.443243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:29:14.443263 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:29:14.443504 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:14.443514 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.445248 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.445252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:14.447335 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:14.447346 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.448881 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.448887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:14.449017 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:14.449061 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 18:29:14.449128 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 18:29:14.449232 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4733,"thread_start_us":128,"threads_started":1}
I20260808 18:29:14.450398 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.431609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60762 (local address 127.7.164.67:34547)
0808 18:29:14.442668 (+ 11059us) server_negotiation.cc:206] Beginning negotiation
0808 18:29:14.442709 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0808 18:29:14.442737 (+ 28us) server_negotiation.cc:407] Connection header received
0808 18:29:14.442819 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:29:14.442824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:29:14.443033 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:29:14.443677 (+ 644us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:29:14.443824 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.445073 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.447663 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.448721 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.449719 (+ 998us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:14.449912 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 18:29:14.449998 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 18:29:14.450187 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10942,"thread_start_us":236,"threads_started":1}
I20260808 18:29:14.442764 24519 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa1880d7bef4a839ecf740b7cce003c (DEFAULT_TABLE table=test-table [id=32e263aac739480096cd99e4dfb55e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:29:14.451110 24519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa1880d7bef4a839ecf740b7cce003c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:29:14.453994 24899 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa1880d7bef4a839ecf740b7cce003c (DEFAULT_TABLE table=test-table [id=32e263aac739480096cd99e4dfb55e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:29:14.455111 24899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa1880d7bef4a839ecf740b7cce003c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:29:14.456867 25035 tablet_bootstrap.cc:492] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb: Bootstrap starting.
I20260808 18:29:14.462694 25036 tablet_bootstrap.cc:492] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b: Bootstrap starting.
I20260808 18:29:14.463136 25035 tablet_bootstrap.cc:654] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb: Neither blocks nor log segments found. Creating new log.
I20260808 18:29:14.464757 25035 log.cc:824] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb: Log is configured to *not* fsync() on all Append() calls
I20260808 18:29:14.465979 25036 tablet_bootstrap.cc:654] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b: Neither blocks nor log segments found. Creating new log.
I20260808 18:29:14.467402 25036 log.cc:824] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b: Log is configured to *not* fsync() on all Append() calls
I20260808 18:29:14.467667 25035 tablet_bootstrap.cc:492] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb: No bootstrap required, opened a new log
I20260808 18:29:14.467875 25035 ts_tablet_manager.cc:1397] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260808 18:29:14.468914 25040 tablet_bootstrap.cc:492] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2: Bootstrap starting.
I20260808 18:29:14.471971 25036 tablet_bootstrap.cc:492] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b: No bootstrap required, opened a new log
I20260808 18:29:14.472262 25036 ts_tablet_manager.cc:1397] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 18:29:14.472297 25040 tablet_bootstrap.cc:654] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2: Neither blocks nor log segments found. Creating new log.
I20260808 18:29:14.473171 25035 raft_consensus.cc:348] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 39829 } } peers { permanent_uuid: "7fea9b08587840e2b031b06e1cf90afb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43923 } } peers { permanent_uuid: "667689b615854325b03a2f876f887ff2" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 34547 } }
I20260808 18:29:14.473469 25035 raft_consensus.cc:374] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:29:14.473538 25035 raft_consensus.cc:729] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fea9b08587840e2b031b06e1cf90afb, State: Initialized, Role: FOLLOWER
I20260808 18:29:14.473558 25040 log.cc:824] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2: Log is configured to *not* fsync() on all Append() calls
I20260808 18:29:14.474164 25035 consensus_queue.cc:260] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 39829 } } peers { permanent_uuid: "7fea9b08587840e2b031b06e1cf90afb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43923 } } peers { permanent_uuid: "667689b615854325b03a2f876f887ff2" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 34547 } }
I20260808 18:29:14.475124 24822 heartbeater.cc:503] Master 127.7.164.126:34305 was elected leader, sending a full tablet report...
I20260808 18:29:14.475672 25035 ts_tablet_manager.cc:1428] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
I20260808 18:29:14.478766 25040 tablet_bootstrap.cc:492] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2: No bootstrap required, opened a new log
I20260808 18:29:14.479100 25040 ts_tablet_manager.cc:1397] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 18:29:14.480249 25036 raft_consensus.cc:348] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 39829 } } peers { permanent_uuid: "7fea9b08587840e2b031b06e1cf90afb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43923 } } peers { permanent_uuid: "667689b615854325b03a2f876f887ff2" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 34547 } }
I20260808 18:29:14.480578 25036 raft_consensus.cc:374] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:29:14.480669 25036 raft_consensus.cc:729] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2aa5a4e5a5a41b9aebfded4c943521b, State: Initialized, Role: FOLLOWER
I20260808 18:29:14.481159 25036 consensus_queue.cc:260] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 39829 } } peers { permanent_uuid: "7fea9b08587840e2b031b06e1cf90afb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43923 } } peers { permanent_uuid: "667689b615854325b03a2f876f887ff2" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 34547 } }
I20260808 18:29:14.481935 24630 heartbeater.cc:503] Master 127.7.164.126:34305 was elected leader, sending a full tablet report...
I20260808 18:29:14.482425 25036 ts_tablet_manager.cc:1428] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b: Time spent starting tablet: real 0.010s user 0.001s sys 0.008s
I20260808 18:29:14.487484 25040 raft_consensus.cc:348] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 39829 } } peers { permanent_uuid: "7fea9b08587840e2b031b06e1cf90afb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43923 } } peers { permanent_uuid: "667689b615854325b03a2f876f887ff2" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 34547 } }
I20260808 18:29:14.487969 25040 raft_consensus.cc:374] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:29:14.488087 25040 raft_consensus.cc:729] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 667689b615854325b03a2f876f887ff2, State: Initialized, Role: FOLLOWER
I20260808 18:29:14.488793 25040 consensus_queue.cc:260] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 39829 } } peers { permanent_uuid: "7fea9b08587840e2b031b06e1cf90afb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43923 } } peers { permanent_uuid: "667689b615854325b03a2f876f887ff2" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 34547 } }
I20260808 18:29:14.489761 25006 heartbeater.cc:503] Master 127.7.164.126:34305 was elected leader, sending a full tablet report...
I20260808 18:29:14.490509 25040 ts_tablet_manager.cc:1428] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
W20260808 18:29:14.535152 24631 tablet.cc:2369] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:29:14.619150 25008 tablet.cc:2369] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 18:29:14 dist-test-slave-jztv krb5kdc[23667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213753, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:29:14.678153 25047 raft_consensus.cc:482] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:29:14.678486 25047 raft_consensus.cc:504] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 39829 } } peers { permanent_uuid: "7fea9b08587840e2b031b06e1cf90afb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43923 } } peers { permanent_uuid: "667689b615854325b03a2f876f887ff2" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 34547 } }
I20260808 18:29:14.680047 25047 leader_election.cc:288] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fea9b08587840e2b031b06e1cf90afb (127.7.164.66:43923), 667689b615854325b03a2f876f887ff2 (127.7.164.67:34547)
I20260808 18:29:14.685523 25025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.680504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:38505 (local address 127.7.164.66:43923)
0808 18:29:14.681165 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0808 18:29:14.681170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:29:14.681188 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:29:14.681248 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:29:14.681252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:29:14.681308 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:29:14.681394 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:29:14.681693 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.682523 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.684196 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.684831 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.684978 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:14.685039 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 18:29:14.685281 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 18:29:14.685375 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560}
I20260808 18:29:14.686112 25050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.680296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:43923 (local address 127.7.164.65:38505)
0808 18:29:14.680665 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 18:29:14.680683 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:29:14.680822 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:29:14.681402 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:29:14.681405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:29:14.681422 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:29:14.681553 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:14.681562 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.682672 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.682675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:14.684054 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:14.684064 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.684998 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.685002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:14.685114 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:14.685142 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 18:29:14.685752 (+ 610us) client_negotiation.cc:240] Negotiation successful
0808 18:29:14.685904 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":154,"threads_started":1}
I20260808 18:29:14.687107 24758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa1880d7bef4a839ecf740b7cce003c" candidate_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fea9b08587840e2b031b06e1cf90afb" is_pre_election: true
I20260808 18:29:14.687531 24758 raft_consensus.cc:2454] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2aa5a4e5a5a41b9aebfded4c943521b in term 0.
I20260808 18:29:14.688112 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.680977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:34693 (local address 127.7.164.67:34547)
0808 18:29:14.681111 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:29:14.681116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:29:14.681171 (+ 55us) server_negotiation.cc:407] Connection header received
0808 18:29:14.681337 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:29:14.681340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:29:14.681405 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:29:14.681478 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:29:14.682668 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.683421 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.686757 (+ 3336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.687420 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.687571 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:14.687625 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:29:14.687893 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 18:29:14.687982 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 18:29:14.688267 24464 leader_election.cc:302] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fea9b08587840e2b031b06e1cf90afb, d2aa5a4e5a5a41b9aebfded4c943521b; no voters:
I20260808 18:29:14.688570 25051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.680797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:34547 (local address 127.7.164.65:34693)
0808 18:29:14.681098 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 18:29:14.681112 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:29:14.681212 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:29:14.681576 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:29:14.681579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:29:14.681599 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:29:14.681844 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:14.681854 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.683851 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.683854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:14.686619 (+ 2765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:29:14.686630 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.687559 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.687563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:29:14.687735 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:14.687758 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:29:14.688336 (+ 578us) client_negotiation.cc:240] Negotiation successful
0808 18:29:14.688436 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":131,"threads_started":1}
I20260808 18:29:14.688627 25047 raft_consensus.cc:2781] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:29:14.688731 25047 raft_consensus.cc:482] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:29:14.688773 25047 raft_consensus.cc:3037] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:29:14.690032 25047 raft_consensus.cc:504] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 39829 } } peers { permanent_uuid: "7fea9b08587840e2b031b06e1cf90afb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43923 } } peers { permanent_uuid: "667689b615854325b03a2f876f887ff2" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 34547 } }
I20260808 18:29:14.690482 25047 leader_election.cc:288] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [CANDIDATE]: Term 1 election: Requested vote from peers 7fea9b08587840e2b031b06e1cf90afb (127.7.164.66:43923), 667689b615854325b03a2f876f887ff2 (127.7.164.67:34547)
I20260808 18:29:14.690953 24938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa1880d7bef4a839ecf740b7cce003c" candidate_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "667689b615854325b03a2f876f887ff2" is_pre_election: true
I20260808 18:29:14.691115 24935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa1880d7bef4a839ecf740b7cce003c" candidate_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "667689b615854325b03a2f876f887ff2"
I20260808 18:29:14.691169 24758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa1880d7bef4a839ecf740b7cce003c" candidate_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fea9b08587840e2b031b06e1cf90afb"
I20260808 18:29:14.691284 24758 raft_consensus.cc:3037] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:29:14.691320 24938 raft_consensus.cc:2454] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2aa5a4e5a5a41b9aebfded4c943521b in term 0.
I20260808 18:29:14.693307 24758 raft_consensus.cc:2454] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2aa5a4e5a5a41b9aebfded4c943521b in term 1.
I20260808 18:29:14.693670 24464 leader_election.cc:302] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fea9b08587840e2b031b06e1cf90afb, d2aa5a4e5a5a41b9aebfded4c943521b; no voters: 667689b615854325b03a2f876f887ff2
I20260808 18:29:14.693850 25047 raft_consensus.cc:2781] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:29:14.694190 25047 raft_consensus.cc:686] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [term 1 LEADER]: Becoming Leader. State: Replica: d2aa5a4e5a5a41b9aebfded4c943521b, State: Running, Role: LEADER
I20260808 18:29:14.694561 25047 consensus_queue.cc:237] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 39829 } } peers { permanent_uuid: "7fea9b08587840e2b031b06e1cf90afb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43923 } } peers { permanent_uuid: "667689b615854325b03a2f876f887ff2" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 34547 } }
I20260808 18:29:14.698467 24317 catalog_manager.cc:5495] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b reported cstate change: term changed from 0 to 1, leader changed from <none> to d2aa5a4e5a5a41b9aebfded4c943521b (127.7.164.65). New cstate: current_term: 1 leader_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2aa5a4e5a5a41b9aebfded4c943521b" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 39829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fea9b08587840e2b031b06e1cf90afb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "667689b615854325b03a2f876f887ff2" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 34547 } health_report { overall_health: UNKNOWN } } }
W20260808 18:29:14.700505 24823 tablet.cc:2369] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:29:14.739506 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.733884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55916 (local address 127.7.164.65:39829)
0808 18:29:14.734250 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0808 18:29:14.734255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:29:14.734270 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:29:14.734363 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:29:14.734366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:29:14.734444 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:29:14.734562 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:29:14.735028 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.735781 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.736914 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:29:14.737279 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:29:14.737384 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:29:14.737811 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:29:14.738467 (+ 656us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:29:14.738624 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 18:29:14.739198 (+ 574us) server_negotiation.cc:299] Negotiation successful
0808 18:29:14.739266 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252}
I20260808 18:29:14.747668 24935 raft_consensus.cc:3037] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:29:14.748376 24758 raft_consensus.cc:1261] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb [term 1 FOLLOWER]: Refusing update from remote peer d2aa5a4e5a5a41b9aebfded4c943521b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:29:14.749265 25047 consensus_queue.cc:1035] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fea9b08587840e2b031b06e1cf90afb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 43923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:29:14.750003 24935 raft_consensus.cc:1261] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2 [term 1 FOLLOWER]: Refusing update from remote peer d2aa5a4e5a5a41b9aebfded4c943521b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:29:14.756206 25047 consensus_queue.cc:1035] T 1aa1880d7bef4a839ecf740b7cce003c P d2aa5a4e5a5a41b9aebfded4c943521b [LEADER]: Connected to new peer: Peer: permanent_uuid: "667689b615854325b03a2f876f887ff2" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:29:14.763453 25056 mvcc.cc:205] Tried to move back new op lower bound from 7316331539438186496 to 7316331539231526912. Current Snapshot: MvccSnapshot[applied={T|T < 7316331539438186496}]
I20260808 18:29:14.766076 25057 mvcc.cc:205] Tried to move back new op lower bound from 7316331539438186496 to 7316331539231526912. Current Snapshot: MvccSnapshot[applied={T|T < 7316331539438186496}]
I20260808 18:29:14.777262 24316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:29:14.777464 24316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:29:14.778718 25059 mvcc.cc:205] Tried to move back new op lower bound from 7316331539438186496 to 7316331539231526912. Current Snapshot: MvccSnapshot[applied={T|T < 7316331539438186496}]
I20260808 18:29:14.780800 24316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f950bdfaafda411ea6d69fabcf2e715a: Sending DeleteTablet for 3 replicas of tablet 1aa1880d7bef4a839ecf740b7cce003c
I20260808 18:29:14.781733 24519 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aa1880d7bef4a839ecf740b7cce003c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:29:14 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:55900
I20260808 18:29:14.782935 24735 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aa1880d7bef4a839ecf740b7cce003c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:29:14 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:37406
I20260808 18:29:14.784101 24899 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aa1880d7bef4a839ecf740b7cce003c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:29:14 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:60762
I20260808 18:29:14.785694 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 24433
I20260808 18:29:14.795293 25072 tablet_replica.cc:330] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2: stopping tablet replica
I20260808 18:29:14.796200 25071 tablet_replica.cc:330] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb: stopping tablet replica
I20260808 18:29:14.795780 25072 raft_consensus.cc:2229] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:29:14.797363 25072 raft_consensus.cc:2258] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:29:14.797782 25071 raft_consensus.cc:2229] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:29:14.798713 25071 raft_consensus.cc:2258] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:29:14.800475 25072 ts_tablet_manager.cc:1905] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:29:14.802631 25071 ts_tablet_manager.cc:1905] T 1aa1880d7bef4a839ecf740b7cce003c P 7fea9b08587840e2b031b06e1cf90afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:29:14.809207 25072 ts_tablet_manager.cc:1918] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:29:14.809465 25072 log.cc:1196] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/1aa1880d7bef4a839ecf740b7cce003c
I20260808 18:29:14.809894 25072 ts_tablet_manager.cc:1939] T 1aa1880d7bef4a839ecf740b7cce003c P 667689b615854325b03a2f876f887ff2: Deleting consensus metadata
I20260808 18:29:14.813997 24299 catalog_manager.cc:4841] TS 667689b615854325b03a2f876f887ff2 (127.7.164.67:34547): tablet 1aa1880d7bef4a839ecf740b7cce003c (table test-table [id=32e263aac739480096cd99e4dfb55e7b]) successfully deleted
W20260808 18:29:14.814844 24299 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:39829 (error 108)
I20260808 18:29:14.815975 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 24641
I20260808 18:29:14.826828 25032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:29:14.826401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:39829 (local address 127.0.0.1:55922)
0808 18:29:14.826631 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 18:29:14.826717 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260808 18:29:14.827224 24299 catalog_manager.cc:4584] TS d2aa5a4e5a5a41b9aebfded4c943521b (127.7.164.65:39829): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1aa1880d7bef4a839ecf740b7cce003c: Network error: Client connection negotiation failed: client connection to 127.7.164.65:39829: connect: Connection refused (error 111)
W20260808 18:29:14.832700 25032 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:29:14.832069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:43923 (local address 127.0.0.1:37420)
0808 18:29:14.832245 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 18:29:14.832260 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:29:14.832353 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:29:14.832582 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.66:43923: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":101}
I20260808 18:29:14.833468 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 24829
I20260808 18:29:14.855373 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 24272
I20260808 18:29:14.867583 7825 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 18:29:15.212661 7825 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 18:29:15.212828 7825 mini_ranger.cc:66] Stopping Ranger...
I20260808 18:29:15.557848 7825 mini_ranger.cc:68] Stopped Ranger
2026-08-08 18:29:15.558 UTC [23702] LOG: received smart shutdown request
2026-08-08 18:29:15.561 UTC [23702] LOG: background worker "logical replication launcher" (PID 23715) exited with exit code 1
2026-08-08 18:29:15.562 UTC [23708] LOG: shutting down
2026-08-08 18:29:15.743 UTC [23702] LOG: database system is shut down
2026-08-08T18:29:15Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50322 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:29:15 dist-test-slave-jztv krb5kdc[25084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:29:15 dist-test-slave-jztv krb5kdc[25084](info): set up 2 sockets
Aug 08 18:29:15 dist-test-slave-jztv krb5kdc[25084](info): commencing operation
krb5kdc: starting...
W20260808 18:29:17.118264 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:29:17 dist-test-slave-jztv krb5kdc[25084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:29:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:29:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.164.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.164.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/rangeradmin_127.7.164.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/rangeradmin_127.7.164.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.164.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.164.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/rangerlookup_127.7.164.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/rangerlookup_127.7.164.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.164.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.164.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/HTTP_127.7.164.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/HTTP_127.7.164.84@KRBTEST.COM.keytab.
I20260808 18:29:17.304260 7825 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasklhTizW/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/postgres -l logfile start
2026-08-08 18:29:19.887 UTC [25126] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 18:29:19.887 UTC [25126] LOG: listening on IPv4 address "127.7.164.84", port 57753
2026-08-08 18:29:19.891 UTC [25126] LOG: listening on Unix socket "/tmp/.s.PGSQL.57753"
2026-08-08 18:29:19.928 UTC [25131] LOG: database system was shut down at 2026-08-08 18:29:18 UTC
2026-08-08 18:29:19.932 UTC [25126] LOG: database system is ready to accept connections
I20260808 18:29:21.070005 7825 mini_postgres.cc:96] Postgres bound to 57753
2026-08-08 18:29:21.077 UTC [25142] FATAL: database "slave" does not exist
127.7.164.84:57753 - accepting connections
I20260808 18:29:21.078173 7825 mini_ranger.cc:161] Starting Ranger...
I20260808 18:29:21.093816 7825 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 18:29:21.600389 7825 mini_ranger.cc:87] Created ranger Postgres database
I20260808 18:29:21.600484 7825 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-admin
2026-08-08 18:29:21,881 [I] DB FLAVOR :POSTGRES
2026-08-08 18:29:21,882 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 18:29:21,882 [I] Checking connection..
2026-08-08 18:29:21,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 18:29:22,142 [I] Checking connection passed.
2026-08-08 18:29:22,142 [I] --------- Verifying version history table ---------
2026-08-08 18:29:22,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:29:22,399 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 18:29:22,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:29:22,662 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 18:29:22,663 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 18:29:22,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 18:29:22.915 UTC [25229] WARNING: there is no transaction in progress
2026-08-08 18:29:22,924 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 18:29:22,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:29:23,191 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 18:29:23,191 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 18:29:23,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 18:29:23,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 18:29:23,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-jztv.c.gcp-upstream.internal','N') ;"
2026-08-08 18:29:23,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 18:29:24,215 [I] Table x_portal_user does not exist in database ranger
2026-08-08 18:29:24,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 18:29:24,468 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 18:29:24,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 18:29:24,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 18:29:24,952 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 18:29:24,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 18:29:25.953 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:25.971 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:25.988 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:26.005 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:26.023 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:26.043 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:26.110 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:26.117 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:26.133 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:26.144 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:26.155 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:26.170 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:26.180 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:26.187 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:26.193 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:26.200 UTC [25381] WARNING: there is no transaction in progress
2026-08-08 18:29:26,895 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 18:29:26,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 18:29:27,149 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 18:29:27,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 18:29:27,411 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 18:29:27,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 18:29:27,663 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 18:29:27,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 18:29:27,917 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 18:29:28,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-jztv.c.gcp-upstream.internal';"
2026-08-08 18:29:28,433 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 18:29:28,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 18:29:28,701 [I] DB_PATCHES have already been applied
I20260808 18:29:28.707408 7825 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-admin:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasklhTizW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 18:29:28.707485 7825 mini_ranger.cc:193] Using host: 127.7.164.84
I20260808 18:29:28.712486 7825 mini_ranger.cc:227] Ranger admin URL: http://127.7.164.84:51981
Aug 08, 2026 6:29:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:29:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:29:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 6:29:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 6:29:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 6:29:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 6:29:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 18:29:30.087399 7825 mini_ranger.cc:160] Time spent starting Ranger: real 9.009s user 0.000s sys 0.012s
Aug 08, 2026 6:29:49 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 6:29:49 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 6:29:49 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 6:29:49 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 6:29:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 6:29:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 6:29:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 6:29:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 6:29:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 6:29:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 6:29:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 6:29:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 6:29:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 6:29:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 6:29:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 6:29:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 6:29:49 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 6:29:50 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 18:29:51.147374 7825 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.164.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.164.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/rangerkms_127.7.164.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.164.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/rangerkms_127.7.164.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.164.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.164.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.164.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/HTTP_127.7.164.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.164.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/HTTP_127.7.164.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 18:29:51 dist-test-slave-jztv krb5kdc[25084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213791, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 18:29:51.234787 7825 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 18:29:51.245610 7825 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 18:29:52.106949 7825 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 18:29:52.107039 7825 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-kms
I20260808 18:29:52.107062 7825 mini_ranger_kms.cc:227] Using postgres at 127.7.164.84:57753
2026-08-08 18:29:52,295 [I] DB FLAVOR :POSTGRES
2026-08-08 18:29:52,295 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 18:29:52,295 [I] Checking connection
2026-08-08 18:29:52,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 18:29:52,533 [I] connection success
2026-08-08 18:29:52,533 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 18:29:52,533 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 18:29:52,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 18:29:52,780 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 18:29:52,780 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 18:29:52,781 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 18:29:52,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.164.84:57753/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 18:29:53,052 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 18:29:53.519232 7825 mini_ranger_kms.cc:310] Created kms service
I20260808 18:29:53.715355 7825 mini_ranger_kms.cc:326] Created kudu user
I20260808 18:29:53.765267 7825 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 18:29:53 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213791, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:29:54.171250 7825 mini_ranger_kms.cc:384] Added ranger policy
I20260808 18:29:54.171540 7825 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-kms:/tmp/dist-test-tasklhTizW/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasklhTizW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 18:29:54.171571 7825 mini_ranger_kms.cc:240] Using host: 127.7.164.84
I20260808 18:29:54.177673 7825 mini_ranger_kms.cc:276] Ranger KMS PID: 25654
I20260808 18:29:54.177755 7825 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.164.84:51235
Aug 08, 2026 6:29:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:29:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:29:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 6:29:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 6:29:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 6:29:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 18:29:56 dist-test-slave-jztv krb5kdc[25084](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213796, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.164.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:29:56.892951 7825 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.658s user 0.017s sys 0.153s
I20260808 18:29:56.893110 7825 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 18:29:56.893186 7825 mini_ranger_kms.cc:401] 127.7.164.84:51235/kms/v1/keys
Aug 08 18:29:57 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213796, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.164.84@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
Aug 08 18:29:57 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213796, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.164.84@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
Aug 08 18:29:59 dist-test-slave-jztv krb5kdc[25084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:29:59.729883 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:41905
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:41905
--ranger_config_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.164.84:51235/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:29:59.869172 25723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:29:59.869732 25723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:29:59.869868 25723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:29:59.876204 25723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:29:59.876374 25723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:29:59.876444 25723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:29:59.876541 25723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:29:59.876636 25723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:29:59.886560 25723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.164.84:51235/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:41905
--ranger_config_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:41905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:29:59.888649 25723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:29:59.890502 25723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:29:59.905867 25729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:29:59.906459 25723 server_base.cc:1034] running on GCE node
W20260808 18:29:59.906664 25731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:29:59.905879 25728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:29:59.907794 25723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:29:59.909683 25723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:29:59.913081 25723 hybrid_clock.cc:648] HybridClock initialized: now 1786213799912999 us; error 104 us; skew 500 ppm
Aug 08 18:29:59 dist-test-slave-jztv krb5kdc[25084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213799, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:29:59.922834 25723 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:29:59.924639 25723 webserver.cc:460] Webserver started at http://127.7.164.126:42005/ using document root <none> and password file <none>
I20260808 18:29:59.925661 25723 fs_manager.cc:359] Metadata directory not provided
I20260808 18:29:59.925750 25723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:29:59.926121 25723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:29:59 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213799, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:30:00.033417 25723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "7e8c83e7005c4063bb5da3b84d89d9df"
format_stamp: "Formatted at 2026-08-08 18:30:00 on dist-test-slave-jztv"
server_key: "7318903d38264fd26598904d0e6b4038"
server_key_iv: "8bfa2ad7dec1984e925eb41960e3503a"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:00.034559 25723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "7e8c83e7005c4063bb5da3b84d89d9df"
format_stamp: "Formatted at 2026-08-08 18:30:00 on dist-test-slave-jztv"
server_key: "7318903d38264fd26598904d0e6b4038"
server_key_iv: "8bfa2ad7dec1984e925eb41960e3503a"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:00.042016 25723 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260808 18:30:00.068372 25739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:00.071193 25723 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260808 18:30:00.071457 25723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "7e8c83e7005c4063bb5da3b84d89d9df"
format_stamp: "Formatted at 2026-08-08 18:30:00 on dist-test-slave-jztv"
server_key: "7318903d38264fd26598904d0e6b4038"
server_key_iv: "8bfa2ad7dec1984e925eb41960e3503a"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:00.071736 25723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:00.122148 25723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:00.133317 25723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:00.133785 25723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:00.147365 25723 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:41905
I20260808 18:30:00.147408 25818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:41905 every 8 connection(s)
I20260808 18:30:00.149097 25723 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:30:00.155682 25821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:00.155944 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 25723
I20260808 18:30:00.156177 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:30:00.156669 7825 external_mini_cluster.cc:1428] Setting key 5932ba17120c65f84fb2ba6724416a12
I20260808 18:30:00.169476 25821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df: Bootstrap starting.
Aug 08 18:30:00 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:00.175057 25821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:00.176753 25821 log.cc:824] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:00.181139 25821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df: No bootstrap required, opened a new log
I20260808 18:30:00.183915 25824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:00.159563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42386 (local address 127.7.164.126:41905)
0808 18:30:00.160260 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:00.160298 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:00.160372 (+ 74us) server_negotiation.cc:407] Connection header received
0808 18:30:00.161838 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:00.161879 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:00.162558 (+ 679us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:00.163312 (+ 754us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:00.164797 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:00.166429 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:00.167743 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:00.168552 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:00.168960 (+ 408us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:00.174138 (+ 5178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:00.174146 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:00.174171 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:00.174313 (+ 142us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:00.178286 (+ 3973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:00.179130 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:00.179151 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:00.179157 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:00.179353 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:00.179895 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:00.179900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:00.179902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:00.180634 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:00.180842 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:00.181351 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 18:30:00.182162 (+ 811us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":232,"threads_started":1}
I20260808 18:30:00.188679 25821 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41905 } }
I20260808 18:30:00.189235 25821 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:00.189455 25821 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e8c83e7005c4063bb5da3b84d89d9df, State: Initialized, Role: FOLLOWER
I20260808 18:30:00.190495 25821 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41905 } }
I20260808 18:30:00.190778 25821 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:30:00.190896 25821 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:30:00.191092 25821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:00.193351 25821 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41905 } }
I20260808 18:30:00.194069 25821 leader_election.cc:302] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e8c83e7005c4063bb5da3b84d89d9df; no voters:
I20260808 18:30:00.194700 25821 leader_election.cc:288] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:30:00.194953 25826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:30:00.195356 25826 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 1 LEADER]: Becoming Leader. State: Replica: 7e8c83e7005c4063bb5da3b84d89d9df, State: Running, Role: LEADER
I20260808 18:30:00.196051 25826 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41905 } }
I20260808 18:30:00.197220 25821 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:30:00.200172 25828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e8c83e7005c4063bb5da3b84d89d9df. Latest consensus state: current_term: 1 leader_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41905 } } }
I20260808 18:30:00.200403 25828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:00.201094 25827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41905 } } }
I20260808 18:30:00.201258 25827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:00.201500 25821 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 18:30:00.202636 25840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:30:00.215654 25840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:30:00.232039 25840 catalog_manager.cc:1347] Generated new cluster ID: 666e8e040645481bbff28544e647c131
I20260808 18:30:00.232160 25840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:30:00.251649 25840 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:30:00.254928 25840 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:30:00.268277 25840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df: Generated new TSK 0
I20260808 18:30:00.269686 25840 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:30:01.498687 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.164.84:51235/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41905
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:01.760172 25876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:01.760833 25876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:01.760973 25876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:01.770972 25876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:01.771129 25876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:01.771266 25876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:30:01.780714 25876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.164.84:51235/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:01.783155 25876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:01.785540 25876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:01.807421 25888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:01.809851 25887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:01.810045 25892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:01.810328 25876 server_base.cc:1034] running on GCE node
I20260808 18:30:01.811107 25876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:01.812178 25876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:01.815626 25876 hybrid_clock.cc:648] HybridClock initialized: now 1786213801815548 us; error 88 us; skew 500 ppm
Aug 08 18:30:01 dist-test-slave-jztv krb5kdc[25084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213801, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:01.820883 25876 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:30:01.822994 25876 webserver.cc:460] Webserver started at http://127.7.164.65:35867/ using document root <none> and password file <none>
I20260808 18:30:01.824129 25876 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:01.824281 25876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:01.824673 25876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:30:01 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213801, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:30:01.862646 25876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "ff150f0d184a4d0fb959fef898d5b306"
format_stamp: "Formatted at 2026-08-08 18:30:01 on dist-test-slave-jztv"
server_key: "39fc790137770b16a4ad8fd1377ae036"
server_key_iv: "7e98551db4f3917c63e29d5653f7b4fa"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:01.864306 25876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "ff150f0d184a4d0fb959fef898d5b306"
format_stamp: "Formatted at 2026-08-08 18:30:01 on dist-test-slave-jztv"
server_key: "39fc790137770b16a4ad8fd1377ae036"
server_key_iv: "7e98551db4f3917c63e29d5653f7b4fa"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:01.876753 25876 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.008s sys 0.000s
I20260808 18:30:01.918995 25906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:01.922185 25876 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.001s
I20260808 18:30:01.922513 25876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "ff150f0d184a4d0fb959fef898d5b306"
format_stamp: "Formatted at 2026-08-08 18:30:01 on dist-test-slave-jztv"
server_key: "39fc790137770b16a4ad8fd1377ae036"
server_key_iv: "7e98551db4f3917c63e29d5653f7b4fa"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:01.922752 25876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:01.972225 25876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:01.979406 25876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:01.979935 25876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:01.981441 25876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:01.983660 25876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:01.983877 25876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 18:30:01.984076 25876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:01.984206 25876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:02.007941 25876 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:45359
I20260808 18:30:02.008021 26067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:45359 every 8 connection(s)
I20260808 18:30:02.010386 25876 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:30:02.018543 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 25876
I20260808 18:30:02.018806 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:30:02.019526 7825 external_mini_cluster.cc:1428] Setting key 13d6532b1d5d213c8e87a5fb1d50ca1c
Aug 08 18:30:02 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213801, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
I20260808 18:30:02.046939 26070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:02.014628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:58087 (local address 127.7.164.126:41905)
0808 18:30:02.015069 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:02.015075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:02.017035 (+ 1960us) server_negotiation.cc:407] Connection header received
0808 18:30:02.019524 (+ 2489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:02.019539 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:02.019671 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:02.019837 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:02.023479 (+ 3642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:02.024837 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:02.028354 (+ 3517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:02.029075 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:02.029285 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:02.037979 (+ 8694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:02.037986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:02.037999 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:02.038072 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:02.041830 (+ 3758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:02.043320 (+ 1490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:02.043327 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:02.043329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:02.043517 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:02.044318 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:02.044321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:02.044322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:02.044759 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:02.044939 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:02.046141 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0808 18:30:02.046518 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":174,"threads_started":1}
I20260808 18:30:02.050649 26072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:02.015583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41905 (local address 127.7.164.65:58087)
0808 18:30:02.016701 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:02.016816 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:02.018945 (+ 2129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:02.020744 (+ 1799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:02.020769 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:02.021538 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:02.023140 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:02.023189 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:02.025194 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:02.025202 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:02.028058 (+ 2856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:02.028086 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:02.029846 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:02.029855 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:02.030446 (+ 591us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:02.033659 (+ 3213us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:02.033729 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:02.037459 (+ 3730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:02.042192 (+ 4733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:02.042204 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:02.042280 (+ 76us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:02.043034 (+ 754us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:02.043730 (+ 696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:02.043735 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:02.043737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:02.044128 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:02.045021 (+ 893us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:02.045028 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:02.045471 (+ 443us) client_negotiation.cc:769] Sending connection context
0808 18:30:02.047256 (+ 1785us) client_negotiation.cc:240] Negotiation successful
0808 18:30:02.048087 (+ 831us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":264,"threads_started":1}
I20260808 18:30:02.053367 26068 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41905
I20260808 18:30:02.053903 26068 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:02.055083 26068 heartbeater.cc:511] Master 127.7.164.126:41905 requested a full tablet report, sending...
I20260808 18:30:02.058465 25762 ts_manager.cc:194] Registered new tserver with Master: ff150f0d184a4d0fb959fef898d5b306 (127.7.164.65:45359)
I20260808 18:30:02.061055 25762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:58087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:30:02.079453 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.164.84:51235/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41905
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:30:02 dist-test-slave-jztv krb5kdc[25084](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 18:30:02.318050 26075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:02.318813 26075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:02.318979 26075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:02.326263 26075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:02.326434 26075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:02.326668 26075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:30:02.337798 26075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.164.84:51235/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:02.340278 26075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:02.342047 26075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:02.361047 26081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:02.361419 26084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:02.361735 26075 server_base.cc:1034] running on GCE node
W20260808 18:30:02.360913 26082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:02.362926 26075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:02.364061 26075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:02.367496 26075 hybrid_clock.cc:648] HybridClock initialized: now 1786213802367412 us; error 89 us; skew 500 ppm
Aug 08 18:30:02 dist-test-slave-jztv krb5kdc[25084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:02.373931 26075 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:30:02.376302 26075 webserver.cc:460] Webserver started at http://127.7.164.66:44347/ using document root <none> and password file <none>
I20260808 18:30:02.377489 26075 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:02.377648 26075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:02.378047 26075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:30:02 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:30:02.409637 26075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "cf2edc10b9be4da1bc85156a59962622"
format_stamp: "Formatted at 2026-08-08 18:30:02 on dist-test-slave-jztv"
server_key: "6c129265e95ca217a33e5775fefa794e"
server_key_iv: "e80d41c40a0b3622facd271f8a09732c"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:02.411792 26075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "cf2edc10b9be4da1bc85156a59962622"
format_stamp: "Formatted at 2026-08-08 18:30:02 on dist-test-slave-jztv"
server_key: "6c129265e95ca217a33e5775fefa794e"
server_key_iv: "e80d41c40a0b3622facd271f8a09732c"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:02.419787 26075 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260808 18:30:02.467545 26095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:02.472955 26075 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.002s sys 0.000s
I20260808 18:30:02.473315 26075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "cf2edc10b9be4da1bc85156a59962622"
format_stamp: "Formatted at 2026-08-08 18:30:02 on dist-test-slave-jztv"
server_key: "6c129265e95ca217a33e5775fefa794e"
server_key_iv: "e80d41c40a0b3622facd271f8a09732c"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:02.473618 26075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:02.506304 26075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:02.511853 26075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:02.512171 26075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:02.513253 26075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:02.514875 26075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:02.515044 26075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:02.515211 26075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:02.515290 26075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:02.538192 26075 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:45861
I20260808 18:30:02.540222 26257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:45861 every 8 connection(s)
I20260808 18:30:02.540490 26075 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:30:02.550709 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 26075
I20260808 18:30:02.550980 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:30:02.551594 7825 external_mini_cluster.cc:1428] Setting key 4638b84fc376883d89147d5fd4d05364
Aug 08 18:30:02 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:02.572832 26070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:02.545274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:43507 (local address 127.7.164.126:41905)
0808 18:30:02.545540 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:02.545555 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:02.549062 (+ 3507us) server_negotiation.cc:407] Connection header received
0808 18:30:02.551444 (+ 2382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:02.551457 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:02.551561 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:02.551679 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:02.555131 (+ 3452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:02.556355 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:02.559533 (+ 3178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:02.560083 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:02.560249 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:02.566080 (+ 5831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:02.566085 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:02.566087 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:02.566133 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:02.568945 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:02.570077 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:02.570085 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:02.570086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:02.570195 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:02.570962 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:02.570966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:02.570967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:02.571280 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:02.571454 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:02.572320 (+ 866us) server_negotiation.cc:299] Negotiation successful
0808 18:30:02.572578 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:30:02.574661 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:02.546193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41905 (local address 127.7.164.66:43507)
0808 18:30:02.548727 (+ 2534us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:02.548851 (+ 124us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:02.550933 (+ 2082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:02.552547 (+ 1614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:02.552559 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:02.553297 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:02.554820 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:02.554857 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:02.556693 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:02.556699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:02.559227 (+ 2528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:02.559242 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:02.560700 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:02.560709 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:02.561186 (+ 477us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:02.562679 (+ 1493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:02.562747 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:02.565739 (+ 2992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:02.569219 (+ 3480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:02.569237 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:02.569268 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:02.569893 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:02.570445 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:02.570452 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:02.570457 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:02.570717 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:02.571442 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:02.571463 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:02.571881 (+ 418us) client_negotiation.cc:769] Sending connection context
0808 18:30:02.572232 (+ 351us) client_negotiation.cc:240] Negotiation successful
0808 18:30:02.572879 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2017,"thread_start_us":298,"threads_started":1}
I20260808 18:30:02.576745 26258 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41905
I20260808 18:30:02.577171 26258 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
I20260808 18:30:02.578181 26258 heartbeater.cc:511] Master 127.7.164.126:41905 requested a full tablet report, sending...
I20260808 18:30:02.580221 25762 ts_manager.cc:194] Registered new tserver with Master: cf2edc10b9be4da1bc85156a59962622 (127.7.164.66:45861)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
I20260808 18:30:02.581314 25762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:43507
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:30:02.611480 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.164.84:51235/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41905
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:02.875667 26264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:02.876168 26264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:02.876313 26264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:02.884011 26264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:02.884317 26264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:02.884554 26264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:30:02.899962 26264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.164.84:51235/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:41905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:02.902374 26264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:02.904116 26264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:02.935505 26276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:02.935590 26275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:02.936383 26278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:02.940425 26264 server_base.cc:1034] running on GCE node
I20260808 18:30:02.941452 26264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:02.942963 26264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:02.946625 26264 hybrid_clock.cc:648] HybridClock initialized: now 1786213802946481 us; error 146 us; skew 500 ppm
Aug 08 18:30:02 dist-test-slave-jztv krb5kdc[25084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:02.953825 26264 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:30:02.956441 26264 webserver.cc:460] Webserver started at http://127.7.164.67:36169/ using document root <none> and password file <none>
I20260808 18:30:02.957856 26264 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:02.958129 26264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:02.958705 26264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:30:02 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
Aug 08 18:30:02 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:30:02.992091 26264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "402c27aa10984247a542dd70ef2f6724"
format_stamp: "Formatted at 2026-08-08 18:30:02 on dist-test-slave-jztv"
server_key: "240bfb1e0af65a29fbe6db84eda50dc9"
server_key_iv: "3d0550b50c102259888bbf53708e95ac"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:02.993089 26264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "402c27aa10984247a542dd70ef2f6724"
format_stamp: "Formatted at 2026-08-08 18:30:02 on dist-test-slave-jztv"
server_key: "240bfb1e0af65a29fbe6db84eda50dc9"
server_key_iv: "3d0550b50c102259888bbf53708e95ac"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:02.999122 26264 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260808 18:30:03.018603 26287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:03.020450 26264 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 18:30:03.020601 26264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "402c27aa10984247a542dd70ef2f6724"
format_stamp: "Formatted at 2026-08-08 18:30:02 on dist-test-slave-jztv"
server_key: "240bfb1e0af65a29fbe6db84eda50dc9"
server_key_iv: "3d0550b50c102259888bbf53708e95ac"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:03.020745 26264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:03.050678 26264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:03.054601 26264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:03.054802 26264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:03.055760 26264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:03.056852 26264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:03.056927 26264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:03.056982 26264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:03.057004 26264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:03.066923 26068 heartbeater.cc:503] Master 127.7.164.126:41905 was elected leader, sending a full tablet report...
I20260808 18:30:03.072780 26264 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:37165
I20260808 18:30:03.075544 26264 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:30:03.080163 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 26264
I20260808 18:30:03.080298 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:30:03.080606 7825 external_mini_cluster.cc:1428] Setting key 0e21d13420dc7003d1ccf1aec78f27e3
I20260808 18:30:03.081948 26452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:37165 every 8 connection(s)
Aug 08 18:30:03 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213802, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:03.101023 26460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:03.085872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:42545 (local address 127.7.164.126:41905)
0808 18:30:03.086747 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:03.086752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:03.086767 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:30:03.088038 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:03.088042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:03.088098 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:03.088199 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:03.089981 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:03.090766 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:03.092571 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:03.092911 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:03.093021 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:03.096371 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:03.096375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:03.096379 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:03.096409 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:03.098449 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:03.099054 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:03.099057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:03.099059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:03.099183 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:03.099644 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:03.099648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:03.099650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:03.099888 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:03.099998 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:03.100697 (+ 699us) server_negotiation.cc:299] Negotiation successful
0808 18:30:03.100840 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":796,"thread_start_us":107,"threads_started":1}
I20260808 18:30:03.102174 26457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:03.085520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41905 (local address 127.7.164.67:42545)
0808 18:30:03.086354 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:03.086419 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:03.087765 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:03.088416 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:03.088425 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:03.088963 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:03.089785 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:03.089807 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:03.090915 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:03.090919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:03.092424 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:03.092436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:03.093083 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:03.093087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:03.093348 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:03.094144 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:03.094165 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:03.096159 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:03.098582 (+ 2423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:03.098588 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:03.098606 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:03.098942 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:03.099295 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:03.099298 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:03.099300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:03.099523 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:03.100022 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:03.100029 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:03.100398 (+ 369us) client_negotiation.cc:769] Sending connection context
0808 18:30:03.100625 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 18:30:03.100978 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":244,"threads_started":1}
I20260808 18:30:03.103570 26454 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41905
I20260808 18:30:03.103863 26454 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:03.104418 26454 heartbeater.cc:511] Master 127.7.164.126:41905 requested a full tablet report, sending...
I20260808 18:30:03.105525 25762 ts_manager.cc:194] Registered new tserver with Master: 402c27aa10984247a542dd70ef2f6724 (127.7.164.67:37165)
I20260808 18:30:03.106112 25762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:42545
I20260808 18:30:03.111598 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:30:03.112237 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 25876
I20260808 18:30:03.132668 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 26075
I20260808 18:30:03.146065 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 26264
I20260808 18:30:03.158756 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 25723
I20260808 18:30:03.174777 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:41905
--webserver_interface=127.7.164.126
--webserver_port=42005
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:41905
--ranger_config_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.164.84:51235/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.164.84:51235/kms with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:03.294592 26462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:03.294936 26462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:03.295027 26462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:03.299221 26462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:30:03.299348 26462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:03.299389 26462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:03.299453 26462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:30:03.299500 26462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:30:03.305315 26462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.164.84:51235/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:41905
--ranger_config_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:41905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=42005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:03.306526 26462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:03.307598 26462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:03.316807 26472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:03.316798 26470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:03.316800 26469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:03.317301 26462 server_base.cc:1034] running on GCE node
I20260808 18:30:03.317879 26462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:03.319151 26462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:03.320336 26462 hybrid_clock.cc:648] HybridClock initialized: now 1786213803320318 us; error 31 us; skew 500 ppm
Aug 08 18:30:03 dist-test-slave-jztv krb5kdc[25084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213803, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:03.323798 26462 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:30:03.324831 26462 webserver.cc:460] Webserver started at http://127.7.164.126:42005/ using document root <none> and password file <none>
I20260808 18:30:03.325377 26462 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:03.325456 26462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 18:30:03 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213803, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:30:03.369016 26462 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260808 18:30:03.371534 26486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:03.372860 26462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 18:30:03.373013 26462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "7e8c83e7005c4063bb5da3b84d89d9df"
format_stamp: "Formatted at 2026-08-08 18:30:00 on dist-test-slave-jztv"
server_key: "7318903d38264fd26598904d0e6b4038"
server_key_iv: "8bfa2ad7dec1984e925eb41960e3503a"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:03.373593 26462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:03.395010 26462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:03.398480 26462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:03.398701 26462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:03.406479 26462 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:41905
I20260808 18:30:03.406509 26574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:41905 every 8 connection(s)
I20260808 18:30:03.407652 26462 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:30:03.412082 26575 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:30:03.413798 26575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df: Bootstrap starting.
I20260808 18:30:03.414747 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 26462
I20260808 18:30:03.415319 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.164.84:51235/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:45359
--local_ip_for_outbound_sockets=127.7.164.65
--tserver_master_addrs=127.7.164.126:41905
--webserver_port=35867
--webserver_interface=127.7.164.65
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.164.84:51235/kms with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:30:03.431551 26575 log.cc:824] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:03.438705 26575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:30:03.439205 26575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df: Bootstrap complete.
I20260808 18:30:03.443615 26575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41905 } }
I20260808 18:30:03.444023 26575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e8c83e7005c4063bb5da3b84d89d9df, State: Initialized, Role: FOLLOWER
I20260808 18:30:03.444553 26575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41905 } }
I20260808 18:30:03.444671 26575 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:30:03.444763 26575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:30:03.444880 26575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:30:03.446985 26575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41905 } }
I20260808 18:30:03.447337 26575 leader_election.cc:302] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e8c83e7005c4063bb5da3b84d89d9df; no voters:
I20260808 18:30:03.447717 26575 leader_election.cc:288] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 18:30:03.447810 26579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 2 FOLLOWER]: Leader election won for term 2
I20260808 18:30:03.448014 26579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [term 2 LEADER]: Becoming Leader. State: Replica: 7e8c83e7005c4063bb5da3b84d89d9df, State: Running, Role: LEADER
I20260808 18:30:03.448320 26579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41905 } }
I20260808 18:30:03.449020 26575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:30:03.449302 26579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e8c83e7005c4063bb5da3b84d89d9df. Latest consensus state: current_term: 2 leader_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41905 } } }
I20260808 18:30:03.449425 26579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:03.449800 26584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:30:03.449832 26580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8c83e7005c4063bb5da3b84d89d9df" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41905 } } }
I20260808 18:30:03.450052 26580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:03.453367 26584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:30:03.453994 26575 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 18:30:03.454068 26584 catalog_manager.cc:1259] Loaded cluster ID: 666e8e040645481bbff28544e647c131
I20260808 18:30:03.454180 26584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:30:03.456622 26584 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:30:03.457312 26584 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df: Loaded TSK: 0
I20260808 18:30:03.458595 26584 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:30:03.567631 26577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:03.568022 26577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:03.568089 26577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:03.574450 26577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:03.574540 26577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:03.574632 26577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:30:03.583308 26577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.164.84:51235/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:45359
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=35867
--tserver_master_addrs=127.7.164.126:41905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:03.584882 26577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:03.586287 26577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:03.599871 26623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:03.600236 26577 server_base.cc:1034] running on GCE node
W20260808 18:30:03.600256 26622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:03.599936 26626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:03.600919 26577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:03.601742 26577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:03.602952 26577 hybrid_clock.cc:648] HybridClock initialized: now 1786213803602935 us; error 42 us; skew 500 ppm
Aug 08 18:30:03 dist-test-slave-jztv krb5kdc[25084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213803, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:03.606739 26577 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:30:03.607889 26577 webserver.cc:460] Webserver started at http://127.7.164.65:35867/ using document root <none> and password file <none>
I20260808 18:30:03.608614 26577 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:03.608711 26577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 18:30:03 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213803, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:30:03.641808 26577 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:30:03.651744 26638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:03.653717 26577 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.000s sys 0.002s
I20260808 18:30:03.653906 26577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "ff150f0d184a4d0fb959fef898d5b306"
format_stamp: "Formatted at 2026-08-08 18:30:01 on dist-test-slave-jztv"
server_key: "39fc790137770b16a4ad8fd1377ae036"
server_key_iv: "7e98551db4f3917c63e29d5653f7b4fa"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:03.654765 26577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:03.683269 26577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:03.687296 26577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:03.687546 26577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:03.688355 26577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:03.689529 26577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:03.689599 26577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:03.689661 26577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:03.689694 26577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:03.701308 26577 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:45359
I20260808 18:30:03.703315 26812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:45359 every 8 connection(s)
I20260808 18:30:03.704336 26577 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:30:03.709142 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 26577
I20260808 18:30:03.709625 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.164.84:51235/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:45861
--local_ip_for_outbound_sockets=127.7.164.66
--tserver_master_addrs=127.7.164.126:41905
--webserver_port=44347
--webserver_interface=127.7.164.66
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.164.84:51235/kms with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:30:03 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213803, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:03.736948 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:03.706995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41905 (local address 127.7.164.65:41529)
0808 18:30:03.707626 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:03.707684 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:03.708730 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:03.710596 (+ 1866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:03.710605 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:03.710952 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:03.711651 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:03.711672 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:03.717174 (+ 5502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:03.717178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:03.718690 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:03.718706 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:03.719862 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:03.719866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:03.720072 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:03.720904 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:03.720930 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:03.725654 (+ 4724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:03.729008 (+ 3354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:03.729017 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:03.729033 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:03.729405 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:03.734010 (+ 4605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:03.734020 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:03.734022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:03.734192 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:03.734958 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:03.734966 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:03.735275 (+ 309us) client_negotiation.cc:769] Sending connection context
0808 18:30:03.735477 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 18:30:03.735870 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":158,"threads_started":1}
I20260808 18:30:03.736928 26821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:03.708572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:41529 (local address 127.7.164.126:41905)
0808 18:30:03.709241 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:03.709265 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:03.709307 (+ 42us) server_negotiation.cc:407] Connection header received
0808 18:30:03.709620 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:03.709635 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:03.710065 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:03.711903 (+ 1838us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:03.712238 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:03.716978 (+ 4740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:03.718872 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:03.719657 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:03.726467 (+ 6810us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:03.726722 (+ 255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:03.726728 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:03.726750 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:03.726780 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:03.728851 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:03.733150 (+ 4299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:03.733158 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:03.733165 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:03.733354 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:03.734353 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:03.734357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:03.734359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:03.734808 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:03.734962 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:03.735619 (+ 657us) server_negotiation.cc:299] Negotiation successful
0808 18:30:03.735891 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":126,"threads_started":1}
I20260808 18:30:03.740276 26813 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41905
I20260808 18:30:03.740561 26813 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:03.741086 26813 heartbeater.cc:511] Master 127.7.164.126:41905 requested a full tablet report, sending...
W20260808 18:30:03.743199 26813 heartbeater.cc:644] Failed to heartbeat to 127.7.164.126:41905 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 18:30:03.743336 26813 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:03.743515 26813 heartbeater.cc:511] Master 127.7.164.126:41905 requested a full tablet report, sending...
I20260808 18:30:03.744328 26813 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:03.744524 26813 heartbeater.cc:511] Master 127.7.164.126:41905 requested a full tablet report, sending...
W20260808 18:30:03.745290 26813 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:30:03.888893 26822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:03.889323 26822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:03.889421 26822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:03.896299 26822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:03.896406 26822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:03.896512 26822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:30:03.905355 26822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.164.84:51235/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:45861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=44347
--tserver_master_addrs=127.7.164.126:41905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:03.906946 26822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:03.908363 26822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:03.922156 26828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:03.922765 26830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:03.923000 26827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:03.923822 26822 server_base.cc:1034] running on GCE node
I20260808 18:30:03.924351 26822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:03.925171 26822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:03.928464 26822 hybrid_clock.cc:648] HybridClock initialized: now 1786213803928384 us; error 93 us; skew 500 ppm
Aug 08 18:30:03 dist-test-slave-jztv krb5kdc[25084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213803, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:03.931663 26822 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:30:03.932766 26822 webserver.cc:460] Webserver started at http://127.7.164.66:44347/ using document root <none> and password file <none>
I20260808 18:30:03.933521 26822 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:03.933640 26822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 18:30:03 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213803, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:30:03.964813 26822 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:30:03.971215 26846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:03.973086 26822 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260808 18:30:03.973225 26822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "cf2edc10b9be4da1bc85156a59962622"
format_stamp: "Formatted at 2026-08-08 18:30:02 on dist-test-slave-jztv"
server_key: "6c129265e95ca217a33e5775fefa794e"
server_key_iv: "e80d41c40a0b3622facd271f8a09732c"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:03.975060 26822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:04.004453 26822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:04.008055 26822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:04.008284 26822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:04.009064 26822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:04.010089 26822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:04.010157 26822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:04.010255 26822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:04.010291 26822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:04.021818 26822 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:45861
I20260808 18:30:04.021836 27013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:45861 every 8 connection(s)
I20260808 18:30:04.023239 26822 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
Aug 08 18:30:04 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213803, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:04.033468 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 26822
I20260808 18:30:04.034055 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.164.84:51235/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:37165
--local_ip_for_outbound_sockets=127.7.164.67
--tserver_master_addrs=127.7.164.126:41905
--webserver_port=36169
--webserver_interface=127.7.164.67
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.164.84:51235/kms with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:30:04.038012 26821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:04.025319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:43501 (local address 127.7.164.126:41905)
0808 18:30:04.025458 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:04.025463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:04.026780 (+ 1317us) server_negotiation.cc:407] Connection header received
0808 18:30:04.027743 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:04.027747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:04.027805 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:04.027914 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:04.029133 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:04.029676 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:04.031003 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:04.031259 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:04.031377 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:04.033948 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:04.033951 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:04.033953 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:04.033983 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:04.035611 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:04.036172 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:04.036176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:04.036177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:04.036286 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:04.036654 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:04.036658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:04.036660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:04.036858 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:04.036977 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:04.037678 (+ 701us) server_negotiation.cc:299] Negotiation successful
0808 18:30:04.037815 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:30:04.038955 27018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:04.025857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41905 (local address 127.7.164.66:43501)
0808 18:30:04.026582 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:04.026636 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:04.027519 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:04.028079 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:04.028085 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:04.028387 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:04.028969 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:04.028984 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:04.029818 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:04.029822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:04.030872 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:04.030883 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:04.031424 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:04.031429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:04.031598 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:04.032254 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:04.032270 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:04.033754 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:04.035742 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:04.035748 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:04.035764 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:04.036045 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:04.036385 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:04.036389 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:04.036391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:04.036551 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:04.037178 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:04.037183 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:04.037433 (+ 250us) client_negotiation.cc:769] Sending connection context
0808 18:30:04.037636 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 18:30:04.037975 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":147,"threads_started":1}
I20260808 18:30:04.040225 27014 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41905
I20260808 18:30:04.040515 27014 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:04.041052 27014 heartbeater.cc:511] Master 127.7.164.126:41905 requested a full tablet report, sending...
W20260808 18:30:04.042636 27014 heartbeater.cc:644] Failed to heartbeat to 127.7.164.126:41905 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 18:30:04.042773 27014 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:04.042970 27014 heartbeater.cc:511] Master 127.7.164.126:41905 requested a full tablet report, sending...
I20260808 18:30:04.043798 27014 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:04.043991 27014 heartbeater.cc:511] Master 127.7.164.126:41905 requested a full tablet report, sending...
W20260808 18:30:04.044749 27014 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:30:04.194020 27019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:04.194418 27019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:04.194507 27019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:04.199350 27019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:04.199417 27019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:04.199510 27019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:30:04.205397 27019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.164.84:51235/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:37165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=36169
--tserver_master_addrs=127.7.164.126:41905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:04.206977 27019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:04.208316 27019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:04.219981 27032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:04.220369 27026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:04.222038 27029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:04.223532 27019 server_base.cc:1034] running on GCE node
I20260808 18:30:04.224004 27019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:04.224819 27019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:04.226033 27019 hybrid_clock.cc:648] HybridClock initialized: now 1786213804225997 us; error 55 us; skew 500 ppm
Aug 08 18:30:04 dist-test-slave-jztv krb5kdc[25084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213804, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:04.229606 27019 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:30:04.230873 27019 webserver.cc:460] Webserver started at http://127.7.164.67:36169/ using document root <none> and password file <none>
I20260808 18:30:04.231647 27019 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:04.231813 27019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 18:30:04 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213804, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:30:04.248981 27019 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:30:04.252264 27043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:04.253963 27019 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260808 18:30:04.254094 27019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "402c27aa10984247a542dd70ef2f6724"
format_stamp: "Formatted at 2026-08-08 18:30:02 on dist-test-slave-jztv"
server_key: "240bfb1e0af65a29fbe6db84eda50dc9"
server_key_iv: "3d0550b50c102259888bbf53708e95ac"
server_key_version: "kuduclusterkey@0"
I20260808 18:30:04.254864 27019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:04.273412 27019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:04.277432 27019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:04.277654 27019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:04.278519 27019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:04.279758 27019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:04.279825 27019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:04.279894 27019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:04.279938 27019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:04.294390 27219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:37165 every 8 connection(s)
I20260808 18:30:04.294647 27019 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:37165
I20260808 18:30:04.295994 27019 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:30:04.305898 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 27019
Aug 08 18:30:04 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213804, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:04.311717 26821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:04.298007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:54809 (local address 127.7.164.126:41905)
0808 18:30:04.298165 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:04.298170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:04.299285 (+ 1115us) server_negotiation.cc:407] Connection header received
0808 18:30:04.300507 (+ 1222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:04.300510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:04.300559 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:04.300635 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:04.302099 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:04.302691 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:04.304419 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:04.304657 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:04.304745 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:04.307715 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:04.307719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:04.307721 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:04.307746 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:04.309387 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:04.310040 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:04.310044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:04.310046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:04.310142 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:04.310573 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:04.310576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:04.310577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:04.310764 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:04.310872 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:04.311404 (+ 532us) server_negotiation.cc:299] Negotiation successful
0808 18:30:04.311539 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 18:30:04.312717 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:04.298530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41905 (local address 127.7.164.67:54809)
0808 18:30:04.299094 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:04.299155 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:04.300268 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:04.300837 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:04.300847 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:04.301233 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:04.301910 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:04.301946 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:04.302838 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:04.302841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:04.304278 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:04.304291 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:04.304824 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:04.304828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:04.305047 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:04.305748 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:04.305770 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:04.307531 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:04.309543 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:04.309550 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:04.309568 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:04.309916 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:04.310261 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:04.310264 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:04.310266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:04.310463 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:04.310872 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:04.310881 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:04.311151 (+ 270us) client_negotiation.cc:769] Sending connection context
0808 18:30:04.311345 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 18:30:04.311676 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":149,"threads_started":1}
I20260808 18:30:04.314051 27222 heartbeater.cc:347] Connected to a master server at 127.7.164.126:41905
I20260808 18:30:04.314332 27222 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:04.314980 27222 heartbeater.cc:511] Master 127.7.164.126:41905 requested a full tablet report, sending...
I20260808 18:30:04.316531 26513 ts_manager.cc:194] Registered new tserver with Master: 402c27aa10984247a542dd70ef2f6724 (127.7.164.67:37165)
I20260808 18:30:04.318017 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:04.306707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42390 (local address 127.7.164.126:41905)
0808 18:30:04.307054 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:04.307064 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:04.307896 (+ 832us) server_negotiation.cc:407] Connection header received
0808 18:30:04.308341 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:04.308347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:04.308409 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:04.308521 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:04.309478 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:04.310282 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:04.311877 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:04.312188 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:04.312291 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:04.314259 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:04.314262 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:04.314264 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:04.314288 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:04.315967 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:04.316525 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:04.316528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:04.316530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:04.316648 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:04.316987 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:04.316991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:04.316993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:04.317223 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:04.317385 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:04.317743 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 18:30:04.317856 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":135,"threads_started":1}
I20260808 18:30:04.318085 26513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:54809
Aug 08 18:30:04 dist-test-slave-jztv krb5kdc[25084](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213804, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:04.745556 26813 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:04.745882 26813 heartbeater.cc:511] Master 127.7.164.126:41905 requested a full tablet report, sending...
I20260808 18:30:04.746663 26513 ts_manager.cc:194] Registered new tserver with Master: ff150f0d184a4d0fb959fef898d5b306 (127.7.164.65:45359)
I20260808 18:30:04.747493 26513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:41529
Aug 08 18:30:05 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213804, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:30:05.045024 27014 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:05.045298 27014 heartbeater.cc:511] Master 127.7.164.126:41905 requested a full tablet report, sending...
I20260808 18:30:05.046128 26513 ts_manager.cc:194] Registered new tserver with Master: cf2edc10b9be4da1bc85156a59962622 (127.7.164.66:45861)
I20260808 18:30:05.046929 26513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:43501
I20260808 18:30:05.056012 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:30:05.091887 27252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:05.058354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42392 (local address 127.7.164.126:41905)
0808 18:30:05.063507 (+ 5153us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:05.063512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:05.063531 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:30:05.063585 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:05.063588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:05.063654 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:05.063758 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:05.064501 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.074663 (+ 10162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.082265 (+ 7602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.082654 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.083267 (+ 613us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:05.087704 (+ 4437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:05.087709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:05.087712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:05.087751 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:05.089671 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:05.090258 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:05.090262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:05.090264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:05.090360 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:05.090663 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:05.090666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:05.090668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:05.090900 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:05.091034 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:05.091391 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 18:30:05.091534 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5014,"thread_start_us":63,"threads_started":1}
I20260808 18:30:05.096501 26513 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:30:05.101446 26513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:30:05.124444 27259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:05.115160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:45359 (local address 127.0.0.1:51464)
0808 18:30:05.117168 (+ 2008us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:05.117179 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:05.117244 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:05.118930 (+ 1686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:05.118934 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:05.118953 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:05.119048 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:05.119054 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.120536 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.120539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:05.122529 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:05.122539 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.123860 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.123864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:05.124004 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:05.124047 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 18:30:05.124159 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 18:30:05.124280 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1959,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260808 18:30:05.125871 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:05.115374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51464 (local address 127.7.164.65:45359)
0808 18:30:05.115987 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:05.115995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:05.117333 (+ 1338us) server_negotiation.cc:407] Connection header received
0808 18:30:05.117415 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:05.117420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:05.117609 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:05.117739 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:05.119176 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.120388 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.122670 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.123532 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.123638 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:05.123797 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:05.124807 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0808 18:30:05.125725 (+ 918us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":143,"threads_started":1}
I20260808 18:30:05.126766 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:05.113565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:37165 (local address 127.0.0.1:38288)
0808 18:30:05.116898 (+ 3333us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:05.116923 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:05.117012 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:05.120668 (+ 3656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:05.120671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:05.120689 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:05.120773 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:05.120779 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.122158 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.122162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:05.123363 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:05.123375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.126389 (+ 3014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.126393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:05.126491 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:05.126514 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:30:05.126579 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 18:30:05.126643 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3267,"thread_start_us":122,"threads_started":1}
I20260808 18:30:05.125129 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:05.113471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:45861 (local address 127.0.0.1:50116)
0808 18:30:05.115788 (+ 2317us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:05.115840 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:05.115969 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:05.116674 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:05.116681 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:05.116709 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:05.116873 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:05.116882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.118441 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.118444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:05.119862 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:05.119874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.124722 (+ 4848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.124727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:05.124824 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:05.124846 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 18:30:05.124916 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 18:30:05.124982 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2117,"mutex_wait_us":3,"thread_start_us":114,"threads_started":1}
I20260808 18:30:05.127595 27264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:05.114365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50116 (local address 127.7.164.66:45861)
0808 18:30:05.116165 (+ 1800us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:05.116171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:05.116194 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:30:05.116270 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:05.116275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:05.116486 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:05.116641 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:05.117039 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.118302 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.123529 (+ 5227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.124392 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.124510 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:05.124660 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:05.127339 (+ 2679us) server_negotiation.cc:299] Negotiation successful
0808 18:30:05.127459 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1704,"thread_start_us":111,"threads_started":1}
I20260808 18:30:05.127913 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:05.114361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38288 (local address 127.7.164.67:37165)
0808 18:30:05.116259 (+ 1898us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:05.116265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:05.120027 (+ 3762us) server_negotiation.cc:407] Connection header received
0808 18:30:05.120119 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:05.120124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:05.120355 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:05.120515 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:05.120903 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.122019 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.125220 (+ 3201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.126068 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.126179 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:05.127616 (+ 1437us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:05.127672 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 18:30:05.127770 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1792,"thread_start_us":109,"threads_started":1}
I20260808 18:30:05.129827 26709 tablet_service.cc:1467] Processing CreateTablet for tablet 627fb080c96f428a86a88b87964dfa7e (DEFAULT_TABLE table=test-table [id=0a3c0b98749b48f1bc0f7e697609a6b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:05.130542 26893 tablet_service.cc:1467] Processing CreateTablet for tablet 627fb080c96f428a86a88b87964dfa7e (DEFAULT_TABLE table=test-table [id=0a3c0b98749b48f1bc0f7e697609a6b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:05.130755 27118 tablet_service.cc:1467] Processing CreateTablet for tablet 627fb080c96f428a86a88b87964dfa7e (DEFAULT_TABLE table=test-table [id=0a3c0b98749b48f1bc0f7e697609a6b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:05.131544 26709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627fb080c96f428a86a88b87964dfa7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:05.131544 26893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627fb080c96f428a86a88b87964dfa7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:05.131747 27118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627fb080c96f428a86a88b87964dfa7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:05.144037 27271 tablet_bootstrap.cc:492] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: Bootstrap starting.
I20260808 18:30:05.147816 27271 tablet_bootstrap.cc:654] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:05.149000 27271 log.cc:824] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:05.150144 27268 tablet_bootstrap.cc:492] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306: Bootstrap starting.
I20260808 18:30:05.151940 27271 tablet_bootstrap.cc:492] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: No bootstrap required, opened a new log
I20260808 18:30:05.152302 27271 ts_tablet_manager.cc:1397] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 18:30:05.152866 27268 tablet_bootstrap.cc:654] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:05.153362 27269 tablet_bootstrap.cc:492] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724: Bootstrap starting.
I20260808 18:30:05.157752 27271 raft_consensus.cc:348] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2edc10b9be4da1bc85156a59962622" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45861 } } peers { permanent_uuid: "402c27aa10984247a542dd70ef2f6724" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37165 } } peers { permanent_uuid: "ff150f0d184a4d0fb959fef898d5b306" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 45359 } }
I20260808 18:30:05.157974 27269 tablet_bootstrap.cc:654] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:05.158174 27271 raft_consensus.cc:374] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:05.158303 27271 raft_consensus.cc:729] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf2edc10b9be4da1bc85156a59962622, State: Initialized, Role: FOLLOWER
I20260808 18:30:05.158833 27269 log.cc:824] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:05.158862 27271 consensus_queue.cc:260] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2edc10b9be4da1bc85156a59962622" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45861 } } peers { permanent_uuid: "402c27aa10984247a542dd70ef2f6724" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37165 } } peers { permanent_uuid: "ff150f0d184a4d0fb959fef898d5b306" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 45359 } }
I20260808 18:30:05.159664 27268 log.cc:824] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:05.160163 27271 ts_tablet_manager.cc:1428] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260808 18:30:05.160574 27014 heartbeater.cc:503] Master 127.7.164.126:41905 was elected leader, sending a full tablet report...
I20260808 18:30:05.162292 27269 tablet_bootstrap.cc:492] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724: No bootstrap required, opened a new log
I20260808 18:30:05.162503 27269 ts_tablet_manager.cc:1397] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 18:30:05.167848 27269 raft_consensus.cc:348] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2edc10b9be4da1bc85156a59962622" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45861 } } peers { permanent_uuid: "402c27aa10984247a542dd70ef2f6724" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37165 } } peers { permanent_uuid: "ff150f0d184a4d0fb959fef898d5b306" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 45359 } }
I20260808 18:30:05.168300 27269 raft_consensus.cc:374] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:05.168407 27269 raft_consensus.cc:729] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 402c27aa10984247a542dd70ef2f6724, State: Initialized, Role: FOLLOWER
I20260808 18:30:05.169005 27269 consensus_queue.cc:260] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2edc10b9be4da1bc85156a59962622" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45861 } } peers { permanent_uuid: "402c27aa10984247a542dd70ef2f6724" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37165 } } peers { permanent_uuid: "ff150f0d184a4d0fb959fef898d5b306" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 45359 } }
I20260808 18:30:05.170181 27269 ts_tablet_manager.cc:1428] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260808 18:30:05.170306 27268 tablet_bootstrap.cc:492] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306: No bootstrap required, opened a new log
I20260808 18:30:05.170568 27268 ts_tablet_manager.cc:1397] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.000s
I20260808 18:30:05.170696 27222 heartbeater.cc:503] Master 127.7.164.126:41905 was elected leader, sending a full tablet report...
I20260808 18:30:05.175999 27268 raft_consensus.cc:348] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2edc10b9be4da1bc85156a59962622" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45861 } } peers { permanent_uuid: "402c27aa10984247a542dd70ef2f6724" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37165 } } peers { permanent_uuid: "ff150f0d184a4d0fb959fef898d5b306" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 45359 } }
I20260808 18:30:05.176337 27268 raft_consensus.cc:374] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:05.176460 27268 raft_consensus.cc:729] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff150f0d184a4d0fb959fef898d5b306, State: Initialized, Role: FOLLOWER
I20260808 18:30:05.177078 27268 consensus_queue.cc:260] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2edc10b9be4da1bc85156a59962622" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45861 } } peers { permanent_uuid: "402c27aa10984247a542dd70ef2f6724" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37165 } } peers { permanent_uuid: "ff150f0d184a4d0fb959fef898d5b306" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 45359 } }
I20260808 18:30:05.177907 26813 heartbeater.cc:503] Master 127.7.164.126:41905 was elected leader, sending a full tablet report...
I20260808 18:30:05.180120 27268 ts_tablet_manager.cc:1428] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
W20260808 18:30:05.206489 26817 tablet.cc:2369] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:30:05.246367 27281 raft_consensus.cc:482] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:30:05.246619 27281 raft_consensus.cc:504] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2edc10b9be4da1bc85156a59962622" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45861 } } peers { permanent_uuid: "402c27aa10984247a542dd70ef2f6724" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37165 } } peers { permanent_uuid: "ff150f0d184a4d0fb959fef898d5b306" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 45359 } }
I20260808 18:30:05.248741 27281 leader_election.cc:288] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf2edc10b9be4da1bc85156a59962622 (127.7.164.66:45861), 402c27aa10984247a542dd70ef2f6724 (127.7.164.67:37165)
I20260808 18:30:05.255043 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:05.249146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:53805 (local address 127.7.164.67:37165)
0808 18:30:05.249263 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:05.249268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:05.249898 (+ 630us) server_negotiation.cc:407] Connection header received
0808 18:30:05.250141 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:05.250145 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:05.250280 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:05.250344 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:05.251142 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.251924 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.253654 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.254296 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.254406 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:05.254463 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:05.254828 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 18:30:05.254914 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:30:05.255693 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:05.248975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:37165 (local address 127.7.164.65:53805)
0808 18:30:05.249780 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:05.249794 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:05.249980 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:05.250744 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:05.250747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:05.250771 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:05.250980 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:05.250991 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.252075 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.252077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:05.253361 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:05.253373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.254567 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.254570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:05.254680 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:05.254703 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:30:05.255461 (+ 758us) client_negotiation.cc:240] Negotiation successful
0808 18:30:05.255542 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":371,"thread_start_us":121,"threads_started":1}
I20260808 18:30:05.256537 27125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627fb080c96f428a86a88b87964dfa7e" candidate_uuid: "ff150f0d184a4d0fb959fef898d5b306" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "402c27aa10984247a542dd70ef2f6724" is_pre_election: true
I20260808 18:30:05.256865 27125 raft_consensus.cc:2454] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff150f0d184a4d0fb959fef898d5b306 in term 0.
I20260808 18:30:05.257526 26640 leader_election.cc:302] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 402c27aa10984247a542dd70ef2f6724, ff150f0d184a4d0fb959fef898d5b306; no voters:
I20260808 18:30:05.257905 27281 raft_consensus.cc:2781] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:30:05.257987 27281 raft_consensus.cc:482] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:30:05.258023 27281 raft_consensus.cc:3037] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:05.259718 27281 raft_consensus.cc:504] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2edc10b9be4da1bc85156a59962622" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45861 } } peers { permanent_uuid: "402c27aa10984247a542dd70ef2f6724" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37165 } } peers { permanent_uuid: "ff150f0d184a4d0fb959fef898d5b306" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 45359 } }
I20260808 18:30:05.260608 27125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627fb080c96f428a86a88b87964dfa7e" candidate_uuid: "ff150f0d184a4d0fb959fef898d5b306" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "402c27aa10984247a542dd70ef2f6724"
I20260808 18:30:05.260725 27125 raft_consensus.cc:3037] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:05.262574 27125 raft_consensus.cc:2454] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff150f0d184a4d0fb959fef898d5b306 in term 1.
I20260808 18:30:05.263021 26640 leader_election.cc:302] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 402c27aa10984247a542dd70ef2f6724, ff150f0d184a4d0fb959fef898d5b306; no voters:
I20260808 18:30:05.263338 27281 leader_election.cc:288] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [CANDIDATE]: Term 1 election: Requested vote from peers cf2edc10b9be4da1bc85156a59962622 (127.7.164.66:45861), 402c27aa10984247a542dd70ef2f6724 (127.7.164.67:37165)
I20260808 18:30:05.263468 27281 raft_consensus.cc:2781] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:30:05.263572 27281 raft_consensus.cc:686] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [term 1 LEADER]: Becoming Leader. State: Replica: ff150f0d184a4d0fb959fef898d5b306, State: Running, Role: LEADER
I20260808 18:30:05.263895 27264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:05.248471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:60305 (local address 127.7.164.66:45861)
0808 18:30:05.248602 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:05.248606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:05.249166 (+ 560us) server_negotiation.cc:407] Connection header received
0808 18:30:05.249371 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:05.249375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:05.249424 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:05.249471 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:05.250062 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.250887 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.262396 (+ 11509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.263020 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.263186 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:05.263244 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:05.263678 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 18:30:05.263767 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:30:05.264178 27281 consensus_queue.cc:237] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2edc10b9be4da1bc85156a59962622" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45861 } } peers { permanent_uuid: "402c27aa10984247a542dd70ef2f6724" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37165 } } peers { permanent_uuid: "ff150f0d184a4d0fb959fef898d5b306" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 45359 } }
I20260808 18:30:05.267735 26513 catalog_manager.cc:5495] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff150f0d184a4d0fb959fef898d5b306 (127.7.164.65). New cstate: current_term: 1 leader_uuid: "ff150f0d184a4d0fb959fef898d5b306" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2edc10b9be4da1bc85156a59962622" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "402c27aa10984247a542dd70ef2f6724" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff150f0d184a4d0fb959fef898d5b306" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 45359 } health_report { overall_health: HEALTHY } } }
I20260808 18:30:05.270661 27282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:05.248347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:45861 (local address 127.7.164.65:60305)
0808 18:30:05.249069 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:05.249085 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:05.249250 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:05.249660 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:05.249663 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:05.249686 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:05.249843 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:05.249851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.258362 (+ 8511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.258368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:05.259930 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:05.259945 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.263334 (+ 3389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.263338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:05.263509 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:05.263536 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:30:05.264059 (+ 523us) client_negotiation.cc:240] Negotiation successful
0808 18:30:05.264143 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":135,"threads_started":1}
I20260808 18:30:05.272215 26939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627fb080c96f428a86a88b87964dfa7e" candidate_uuid: "ff150f0d184a4d0fb959fef898d5b306" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf2edc10b9be4da1bc85156a59962622" is_pre_election: true
I20260808 18:30:05.272570 26939 raft_consensus.cc:2454] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff150f0d184a4d0fb959fef898d5b306 in term 0.
I20260808 18:30:05.273053 26942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627fb080c96f428a86a88b87964dfa7e" candidate_uuid: "ff150f0d184a4d0fb959fef898d5b306" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf2edc10b9be4da1bc85156a59962622"
I20260808 18:30:05.273171 26942 raft_consensus.cc:3037] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:05.275198 26942 raft_consensus.cc:2454] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff150f0d184a4d0fb959fef898d5b306 in term 1.
W20260808 18:30:05.277494 27017 tablet.cc:2369] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:30:05.297840 27226 tablet.cc:2369] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:30:05.316860 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:05.312227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33690 (local address 127.7.164.65:45359)
0808 18:30:05.312354 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:05.312359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:05.312375 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:30:05.312466 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:05.312469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:05.312555 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:05.312667 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:30:05.313052 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.313591 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.314774 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:05.315061 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:05.315167 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:05.315580 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:30:05.316077 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:30:05.316194 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:05.316628 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 18:30:05.316693 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:30:05.324100 27125 raft_consensus.cc:1261] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724 [term 1 FOLLOWER]: Refusing update from remote peer ff150f0d184a4d0fb959fef898d5b306: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:30:05.324851 27284 consensus_queue.cc:1035] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [LEADER]: Connected to new peer: Peer: permanent_uuid: "402c27aa10984247a542dd70ef2f6724" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 37165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:30:05.325407 26942 raft_consensus.cc:1261] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622 [term 1 FOLLOWER]: Refusing update from remote peer ff150f0d184a4d0fb959fef898d5b306: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:30:05.326771 27284 consensus_queue.cc:1035] T 627fb080c96f428a86a88b87964dfa7e P ff150f0d184a4d0fb959fef898d5b306 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2edc10b9be4da1bc85156a59962622" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 08 18:30:05 dist-test-slave-jztv krb5kdc[25084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213804, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for HTTP/127.7.164.84@KRBTEST.COM
I20260808 18:30:05.335600 27290 mvcc.cc:205] Tried to move back new op lower bound from 7316331746598903808 to 7316331746365616128. Current Snapshot: MvccSnapshot[applied={T|T < 7316331746598903808}]
I20260808 18:30:05.338577 27292 mvcc.cc:205] Tried to move back new op lower bound from 7316331746598903808 to 7316331746365616128. Current Snapshot: MvccSnapshot[applied={T|T < 7316331746598903808}]
I20260808 18:30:05.343818 26509 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:30:05.344013 26509 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:30:05.346832 26509 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e8c83e7005c4063bb5da3b84d89d9df: Sending DeleteTablet for 3 replicas of tablet 627fb080c96f428a86a88b87964dfa7e
I20260808 18:30:05.348462 26893 tablet_service.cc:1514] Processing DeleteTablet for tablet 627fb080c96f428a86a88b87964dfa7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:05 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:50116
I20260808 18:30:05.349124 26709 tablet_service.cc:1514] Processing DeleteTablet for tablet 627fb080c96f428a86a88b87964dfa7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:05 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:51464
I20260808 18:30:05.349618 27118 tablet_service.cc:1514] Processing DeleteTablet for tablet 627fb080c96f428a86a88b87964dfa7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:05 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:38288
I20260808 18:30:05.351370 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 26577
I20260808 18:30:05.352118 27301 tablet_replica.cc:330] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: stopping tablet replica
I20260808 18:30:05.352550 27301 raft_consensus.cc:2229] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:30:05.352883 27301 pending_rounds.cc:62] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: Trying to abort 2 pending ops.
I20260808 18:30:05.352977 27301 pending_rounds.cc:68] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316331746598903808 op_type: WRITE_OP write_request { tablet_id: "627fb080c96f428a86a88b87964dfa7e" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\331\355\335\323\006\"8\n\ntest-admin\022*\n 0a3c0b98749b48f1bc0f7e697609a6b4\020\001\030\001 \001(\001" signature: "<redacted>""Oo\021)\375g\365\354\235\272\264\212P\225@\347\236\332PG\245\321\'\002f\230\337\rM@\350\030\356\200Y<\355\310\033{\325&\350\313\375\025\336\320\237\203\216A\363\361\021P\3324\235\003A:\375\335" signing_key_seq_num: 0 } } request_id { client_id: "786e963e900c4ce2ad4ab8304cf9a1c6" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260808 18:30:05.353358 27301 pending_rounds.cc:68] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316331746365616128 op_type: NO_OP noop_request { }
I20260808 18:30:05.353477 27301 raft_consensus.cc:2866] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260808 18:30:05.353578 27301 raft_consensus.cc:2258] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 18:30:05.364706 27295 consensus_queue.cc:1159] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cf2edc10b9be4da1bc85156a59962622" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260808 18:30:05.369799 27301 ts_tablet_manager.cc:1905] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:30:05.373086 27301 ts_tablet_manager.cc:1918] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:30:05.373219 27301 log.cc:1196] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/627fb080c96f428a86a88b87964dfa7e
I20260808 18:30:05.373574 27301 ts_tablet_manager.cc:1939] T 627fb080c96f428a86a88b87964dfa7e P cf2edc10b9be4da1bc85156a59962622: Deleting consensus metadata
I20260808 18:30:05.375061 27304 tablet_replica.cc:330] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724: stopping tablet replica
I20260808 18:30:05.375205 26488 catalog_manager.cc:4841] TS cf2edc10b9be4da1bc85156a59962622 (127.7.164.66:45861): tablet 627fb080c96f428a86a88b87964dfa7e (table test-table [id=0a3c0b98749b48f1bc0f7e697609a6b4]) successfully deleted
I20260808 18:30:05.375437 27304 raft_consensus.cc:2229] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:30:05.375802 27304 raft_consensus.cc:2258] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:30:05.377540 27304 ts_tablet_manager.cc:1905] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:30:05.380519 27304 ts_tablet_manager.cc:1918] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:30:05.380638 27304 log.cc:1196] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/627fb080c96f428a86a88b87964dfa7e
I20260808 18:30:05.381007 27304 ts_tablet_manager.cc:1939] T 627fb080c96f428a86a88b87964dfa7e P 402c27aa10984247a542dd70ef2f6724: Deleting consensus metadata
I20260808 18:30:05.383992 26487 catalog_manager.cc:4841] TS 402c27aa10984247a542dd70ef2f6724 (127.7.164.67:37165): tablet 627fb080c96f428a86a88b87964dfa7e (table test-table [id=0a3c0b98749b48f1bc0f7e697609a6b4]) successfully deleted
W20260808 18:30:05.384407 26851 connection.cc:535] server connection from 127.7.164.65:60305 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 18:30:05.384925 26488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:45359 (error 108)
I20260808 18:30:05.385964 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 26822
I20260808 18:30:05.403918 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:05.395505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:45359 (local address 127.0.0.1:33694)
0808 18:30:05.403642 (+ 8137us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:05.403779 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:45359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7909}
W20260808 18:30:05.404472 26488 catalog_manager.cc:4584] TS ff150f0d184a4d0fb959fef898d5b306 (127.7.164.65:45359): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 627fb080c96f428a86a88b87964dfa7e: Network error: Client connection negotiation failed: client connection to 127.7.164.65:45359: connect: Connection refused (error 111)
I20260808 18:30:05.410526 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 27019
I20260808 18:30:05.431267 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 26462
I20260808 18:30:05.444914 7825 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 18:30:05.790601 7825 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 18:30:05.790729 7825 mini_ranger.cc:66] Stopping Ranger...
I20260808 18:30:06.135949 7825 mini_ranger.cc:68] Stopped Ranger
2026-08-08 18:30:06.136 UTC [25126] LOG: received smart shutdown request
2026-08-08 18:30:06.140 UTC [25126] LOG: background worker "logical replication launcher" (PID 25138) exited with exit code 1
2026-08-08 18:30:06.140 UTC [25132] LOG: shutting down
2026-08-08 18:30:06.309 UTC [25126] LOG: database system is shut down
2026-08-08T18:30:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50565 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:30:06 dist-test-slave-jztv krb5kdc[27324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:30:06 dist-test-slave-jztv krb5kdc[27324](info): set up 2 sockets
Aug 08 18:30:06 dist-test-slave-jztv krb5kdc[27324](info): commencing operation
krb5kdc: starting...
W20260808 18:30:07.674568 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:30:07 dist-test-slave-jztv krb5kdc[27324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:30:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:30:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:30:07.810122 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:41357
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:41357,127.7.164.125:40863,127.7.164.124:44815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:07.927909 27342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:07.928246 27342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:07.928324 27342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:07.932418 27342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:30:07.932512 27342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:07.932549 27342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:07.932601 27342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:30:07.932639 27342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:30:07.938405 27342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:41357,127.7.164.125:40863,127.7.164.124:44815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:41357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:07.939638 27342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:07.940711 27342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:07.949453 27349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:07.949683 27342 server_base.cc:1034] running on GCE node
W20260808 18:30:07.949429 27354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:07.949453 27350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:07.950501 27342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:07.951567 27342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:07.952755 27342 hybrid_clock.cc:648] HybridClock initialized: now 1786213807952737 us; error 30 us; skew 500 ppm
Aug 08 18:30:07 dist-test-slave-jztv krb5kdc[27324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213807, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:07.955538 27342 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:30:07.956453 27342 webserver.cc:460] Webserver started at http://127.7.164.126:42319/ using document root <none> and password file <none>
I20260808 18:30:07.957016 27342 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:07.957109 27342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:07.957345 27342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:30:07.959444 27342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "0a0ca2f941c64a74afe102fb48681f81"
format_stamp: "Formatted at 2026-08-08 18:30:07 on dist-test-slave-jztv"
server_key: "08264e456b1d0b711529c51be807f613"
server_key_iv: "40f71f2cea5197af7d295d90abd56a77"
server_key_version: "encryptionkey@0"
I20260808 18:30:07.960042 27342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "0a0ca2f941c64a74afe102fb48681f81"
format_stamp: "Formatted at 2026-08-08 18:30:07 on dist-test-slave-jztv"
server_key: "08264e456b1d0b711529c51be807f613"
server_key_iv: "40f71f2cea5197af7d295d90abd56a77"
server_key_version: "encryptionkey@0"
I20260808 18:30:07.963768 27342 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:30:07.966178 27362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:07.967634 27342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:30:07.967741 27342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "0a0ca2f941c64a74afe102fb48681f81"
format_stamp: "Formatted at 2026-08-08 18:30:07 on dist-test-slave-jztv"
server_key: "08264e456b1d0b711529c51be807f613"
server_key_iv: "40f71f2cea5197af7d295d90abd56a77"
server_key_version: "encryptionkey@0"
I20260808 18:30:07.967882 27342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:07.986420 27342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:07.989956 27342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:07.990129 27342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:07.997345 27342 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:41357
I20260808 18:30:07.997426 27430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:41357 every 8 connection(s)
I20260808 18:30:07.998447 27342 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:30:08.001322 27432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:08.005155 27432 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:30:08.008634 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 27342
I20260808 18:30:08.008780 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:30:08.009065 7825 external_mini_cluster.cc:1428] Setting key 220c646f4137215b3f03ef31c22ddc39
Aug 08 18:30:08 dist-test-slave-jztv krb5kdc[27324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213807, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.164.125@KRBTEST.COM; defaulting to no policy
I20260808 18:30:08.021220 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.007539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.7.164.126:41357)
0808 18:30:08.008070 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.008090 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.008290 (+ 200us) server_negotiation.cc:407] Connection header received
0808 18:30:08.009455 (+ 1165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.009459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.009638 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.009830 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.010720 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.012107 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.013149 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.013543 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.013714 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.016357 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.016366 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.016382 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.016403 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.018065 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.018575 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.018581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.018583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.018693 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.019020 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.019025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.019027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.019422 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.019570 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.019936 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.020198 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":352,"thread_start_us":141,"threads_started":1}
I20260808 18:30:08.021216 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.007539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41357 (local address 127.0.0.1:35046)
0808 18:30:08.008140 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.008184 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:08.009145 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:08.009825 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:08.009830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:08.010305 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:08.010558 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.010572 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.012280 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.012283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.013025 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.013035 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.013701 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.013705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.013817 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.014499 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:08.014515 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:08.016204 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:08.018176 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.018187 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.018197 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.018455 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.018795 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.018798 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.018800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.018914 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.019542 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:08.019548 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:08.019699 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 18:30:08.019861 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 18:30:08.020252 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":113,"spinlock_wait_cycles":47360,"thread_start_us":167,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.125@KRBTEST.COM" created.
I20260808 18:30:08.023821 27432 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:30:08.024657 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.024328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:40863 (local address 127.0.0.1:45026)
0808 18:30:08.024475 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.024568 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 18:30:08.024886 27363 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.164.125:40863: connect: Connection refused (error 111)
I20260808 18:30:08.025827 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.025537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:40863 (local address 127.0.0.1:45032)
0808 18:30:08.025682 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.025745 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 18:30:08.025996 27432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.125:40863: Network error: Client connection negotiation failed: client connection to 127.7.164.125:40863: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.125.keytab.
Entry for principal kudu/127.7.164.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.125.keytab.
I20260808 18:30:08.039477 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.125:40863
--webserver_interface=127.7.164.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.125.keytab
--principal=kudu/127.7.164.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:41357,127.7.164.125:40863,127.7.164.124:44815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:30:08.075284 27432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } attempt: 1
I20260808 18:30:08.076277 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.075920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:40863 (local address 127.0.0.1:45042)
0808 18:30:08.076105 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.076182 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 18:30:08.077116 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.076870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:40863 (local address 127.0.0.1:45058)
0808 18:30:08.076992 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.077047 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 18:30:08.077294 27432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.125:40863: Network error: Client connection negotiation failed: client connection to 127.7.164.125:40863: connect: Connection refused (error 111)
I20260808 18:30:08.145517 27432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } attempt: 2
I20260808 18:30:08.146414 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.146034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:40863 (local address 127.0.0.1:45062)
0808 18:30:08.146238 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.146311 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 18:30:08.147156 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.146920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:40863 (local address 127.0.0.1:45078)
0808 18:30:08.147037 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.147092 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 18:30:08.147291 27432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.125:40863: Network error: Client connection negotiation failed: client connection to 127.7.164.125:40863: connect: Connection refused (error 111)
W20260808 18:30:08.157094 27442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:08.157343 27442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:08.157383 27442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:08.161356 27442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:30:08.161415 27442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:08.161434 27442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:08.161450 27442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:30:08.161465 27442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:30:08.166764 27442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:41357,127.7.164.125:40863,127.7.164.124:44815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.125.keytab
--principal=kudu/127.7.164.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.125:40863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.164.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:08.167929 27442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:08.168946 27442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:08.177842 27451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:08.177842 27458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:08.177877 27455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:08.178253 27442 server_base.cc:1034] running on GCE node
I20260808 18:30:08.178889 27442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:08.180096 27442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:08.181284 27442 hybrid_clock.cc:648] HybridClock initialized: now 1786213808181240 us; error 56 us; skew 500 ppm
Aug 08 18:30:08 dist-test-slave-jztv krb5kdc[27324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:08.184448 27442 init.cc:377] Logged in from keytab as kudu/127.7.164.125@KRBTEST.COM (short username kudu)
I20260808 18:30:08.185519 27442 webserver.cc:460] Webserver started at http://127.7.164.125:36017/ using document root <none> and password file <none>
I20260808 18:30:08.186153 27442 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:08.186277 27442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:08.186517 27442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:30:08.188680 27442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/data/instance:
uuid: "b1452adbd170486eb4d8cd5a0da42167"
format_stamp: "Formatted at 2026-08-08 18:30:08 on dist-test-slave-jztv"
server_key: "d47a88671fa12981cecf31fbbfdc6de5"
server_key_iv: "8fbb7c76bf162f582ef12e68e81f1a37"
server_key_version: "encryptionkey@0"
I20260808 18:30:08.189304 27442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/wal/instance:
uuid: "b1452adbd170486eb4d8cd5a0da42167"
format_stamp: "Formatted at 2026-08-08 18:30:08 on dist-test-slave-jztv"
server_key: "d47a88671fa12981cecf31fbbfdc6de5"
server_key_iv: "8fbb7c76bf162f582ef12e68e81f1a37"
server_key_version: "encryptionkey@0"
I20260808 18:30:08.193540 27442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:30:08.196549 27470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:08.198068 27442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:30:08.198261 27442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/wal
uuid: "b1452adbd170486eb4d8cd5a0da42167"
format_stamp: "Formatted at 2026-08-08 18:30:08 on dist-test-slave-jztv"
server_key: "d47a88671fa12981cecf31fbbfdc6de5"
server_key_iv: "8fbb7c76bf162f582ef12e68e81f1a37"
server_key_version: "encryptionkey@0"
I20260808 18:30:08.198400 27442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:08.213636 27442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:08.217120 27442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:08.217373 27442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:08.225039 27442 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.125:40863
I20260808 18:30:08.225116 27536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.125:40863 every 8 connection(s)
I20260808 18:30:08.226092 27442 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/data/info.pb
I20260808 18:30:08.226886 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 27442
I20260808 18:30:08.227034 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-1/wal/instance
I20260808 18:30:08.227332 7825 external_mini_cluster.cc:1428] Setting key fe50a24d358b03abe4e51bd195f647cf
I20260808 18:30:08.229597 27537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:08.234912 27537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.164.124@KRBTEST.COM; defaulting to no policy
I20260808 18:30:08.238431 27432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.124@KRBTEST.COM" created.
Aug 08 18:30:08 dist-test-slave-jztv krb5kdc[27324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213807, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for kudu/127.7.164.125@KRBTEST.COM
Aug 08 18:30:08 dist-test-slave-jztv krb5kdc[27324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.125@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:08.250782 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.238890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:40863 (local address 127.0.0.1:45092)
0808 18:30:08.239121 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.239134 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:08.239215 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:08.240139 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:08.240142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:08.240318 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:08.240441 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.240448 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.242335 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.242339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.242866 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.242874 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.243672 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.243679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.243922 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.244433 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:08.244452 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:08.246242 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:08.248621 (+ 2379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.248625 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.248628 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.248833 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.249251 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.249256 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.249261 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.249355 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.250125 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:08.250128 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:08.250264 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 18:30:08.250417 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 18:30:08.250573 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":43,"spinlock_wait_cycles":2432}
I20260808 18:30:08.251649 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.236950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35056 (local address 127.7.164.126:41357)
0808 18:30:08.237155 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.237160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.238061 (+ 901us) server_negotiation.cc:407] Connection header received
0808 18:30:08.239398 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.239401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.239453 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.239544 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.240998 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.241972 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.243527 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.243960 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.244080 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.247556 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.247563 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.247570 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.247602 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.249656 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.250279 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.250282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.250284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.250378 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.250705 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.250708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.250710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.250935 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.251035 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.251444 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.251543 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":67}
I20260808 18:30:08.252033 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.237320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41357 (local address 127.0.0.1:35056)
0808 18:30:08.237874 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.237942 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:08.239109 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:08.239733 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:08.239740 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:08.240202 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:08.240854 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.240865 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.242130 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.242136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.243386 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.243402 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.244121 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.244400 (+ 279us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.244498 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.245050 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:08.245070 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:08.247125 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:08.249786 (+ 2661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.249798 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.249806 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.250113 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.250484 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.250487 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.250489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.250580 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.251089 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:08.251095 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:08.251211 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 18:30:08.251355 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 18:30:08.251538 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":17,"thread_start_us":129,"threads_started":1}
I20260808 18:30:08.252033 27542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.239118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45092 (local address 127.7.164.125:40863)
0808 18:30:08.239500 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.239515 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.239573 (+ 58us) server_negotiation.cc:407] Connection header received
0808 18:30:08.239699 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.239706 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.239956 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.240146 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.240717 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.242131 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.242995 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.243539 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.243693 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.246374 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.246380 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.246391 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.246414 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.248510 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.248960 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.248967 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.248974 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.249123 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.249488 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.249491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.249493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.250008 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.250162 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.250602 (+ 440us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.250935 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":275,"spinlock_wait_cycles":13056,"thread_start_us":155,"threads_started":1}
I20260808 18:30:08.253732 27537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:30:08.254141 27432 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:30:08.254858 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.254610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39552)
0808 18:30:08.254703 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.254781 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 18:30:08.255640 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.255392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39562)
0808 18:30:08.255530 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.255580 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260808 18:30:08.255791 27432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:44815: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.124.keytab.
Entry for principal kudu/127.7.164.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.124.keytab.
Aug 08 18:30:08 dist-test-slave-jztv krb5kdc[27324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.125@KRBTEST.COM for kudu/127.7.164.125@KRBTEST.COM
I20260808 18:30:08.259438 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.124:44815
--webserver_interface=127.7.164.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.124.keytab
--principal=kudu/127.7.164.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:41357,127.7.164.125:40863,127.7.164.124:44815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:30:08.263110 27542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.254397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45098 (local address 127.7.164.125:40863)
0808 18:30:08.254525 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.254532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.254547 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:30:08.254631 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.254635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.254682 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.254769 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.255200 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.255783 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.256799 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.257090 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.257178 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.259447 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.259451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.259452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.259474 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.260977 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.261614 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.261617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.261618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.261766 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.262113 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.262115 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.262116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.262369 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.262469 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.262856 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.262976 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:30:08.263221 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.254369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:40863 (local address 127.0.0.1:45098)
0808 18:30:08.254466 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.254481 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:08.254562 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:08.254804 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:08.254807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:08.254981 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:08.255088 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.255095 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.255957 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.255961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.256675 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.256686 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.257194 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.257196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.257269 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.257698 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:08.257709 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:08.259289 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:08.261169 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.261173 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.261176 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.261485 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.261882 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.261888 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.261891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.262009 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.262486 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:08.262491 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:08.262618 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 18:30:08.262750 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 18:30:08.262908 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 18:30:08.264107 27537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:30:08.264969 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.264685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39564)
0808 18:30:08.264825 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.264902 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260808 18:30:08.265910 27474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
I20260808 18:30:08.267062 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.266854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39576)
0808 18:30:08.266968 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.267012 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 18:30:08.267328 27537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:44815: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
I20260808 18:30:08.289038 27432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } attempt: 1
I20260808 18:30:08.289974 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.289639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39582)
0808 18:30:08.289794 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.289867 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 18:30:08.290824 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.290563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39590)
0808 18:30:08.290689 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.290749 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 18:30:08.290977 27432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:44815: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
I20260808 18:30:08.316650 27537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } attempt: 1
I20260808 18:30:08.317703 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.317380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39592)
0808 18:30:08.317548 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.317617 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 18:30:08.318537 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.318306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39600)
0808 18:30:08.318425 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.318480 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 18:30:08.318671 27537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:44815: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
I20260808 18:30:08.366150 27432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } attempt: 2
I20260808 18:30:08.367133 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.366822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39616)
0808 18:30:08.366984 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.367049 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 18:30:08.367916 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.367687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39624)
0808 18:30:08.367800 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.367855 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 18:30:08.368054 27432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:44815: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
I20260808 18:30:08.386853 27537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } attempt: 2
I20260808 18:30:08.387702 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.387403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39640)
0808 18:30:08.387550 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.387622 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 18:30:08.388559 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.388281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39648)
0808 18:30:08.388444 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.388495 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 18:30:08.388760 27537 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:44815: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
W20260808 18:30:08.392071 27544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:08.392349 27544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:08.392427 27544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:08.396641 27544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:30:08.396706 27544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:08.396726 27544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:08.396744 27544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:30:08.396803 27544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:30:08.403214 27544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:41357,127.7.164.125:40863,127.7.164.124:44815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.124.keytab
--principal=kudu/127.7.164.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.124:44815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.164.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:08.404419 27544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:08.405498 27544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:30:08.415329 27544 server_base.cc:1034] running on GCE node
W20260808 18:30:08.415266 27551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:08.415347 27553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:08.415266 27550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:08.416132 27544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:08.417276 27544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:08.418468 27544 hybrid_clock.cc:648] HybridClock initialized: now 1786213808418434 us; error 49 us; skew 500 ppm
Aug 08 18:30:08 dist-test-slave-jztv krb5kdc[27324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:08.421449 27544 init.cc:377] Logged in from keytab as kudu/127.7.164.124@KRBTEST.COM (short username kudu)
I20260808 18:30:08.422468 27544 webserver.cc:460] Webserver started at http://127.7.164.124:42419/ using document root <none> and password file <none>
I20260808 18:30:08.423053 27544 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:08.423148 27544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:08.423388 27544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:30:08.425457 27544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/data/instance:
uuid: "7c6556f59abd4f009bfb14288acd1ca9"
format_stamp: "Formatted at 2026-08-08 18:30:08 on dist-test-slave-jztv"
server_key: "1db2896a2d2e540fb0da33658da38f1d"
server_key_iv: "1eeaf2f9a7ce7a496454ae9b7a94bfb8"
server_key_version: "encryptionkey@0"
I20260808 18:30:08.426048 27544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/wal/instance:
uuid: "7c6556f59abd4f009bfb14288acd1ca9"
format_stamp: "Formatted at 2026-08-08 18:30:08 on dist-test-slave-jztv"
server_key: "1db2896a2d2e540fb0da33658da38f1d"
server_key_iv: "1eeaf2f9a7ce7a496454ae9b7a94bfb8"
server_key_version: "encryptionkey@0"
I20260808 18:30:08.429821 27544 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:30:08.432391 27563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:08.433660 27544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:30:08.433821 27544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/wal
uuid: "7c6556f59abd4f009bfb14288acd1ca9"
format_stamp: "Formatted at 2026-08-08 18:30:08 on dist-test-slave-jztv"
server_key: "1db2896a2d2e540fb0da33658da38f1d"
server_key_iv: "1eeaf2f9a7ce7a496454ae9b7a94bfb8"
server_key_version: "encryptionkey@0"
I20260808 18:30:08.433950 27544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:08.464923 27544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:08.467248 27432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } attempt: 3
I20260808 18:30:08.468144 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.467791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39652)
0808 18:30:08.467968 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.468038 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 18:30:08.468737 27544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:08.468943 27544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:08.469111 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.468773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39662)
0808 18:30:08.468959 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.469005 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260808 18:30:08.469298 27432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:44815: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44815: connect: Connection refused (error 111)
I20260808 18:30:08.476647 27544 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.124:44815
I20260808 18:30:08.476675 27642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.124:44815 every 8 connection(s)
I20260808 18:30:08.477708 27544 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/data/info.pb
I20260808 18:30:08.479931 27537 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } attempt: 3
I20260808 18:30:08.480880 27644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:08.482513 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 27544
I20260808 18:30:08.482656 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213640155572-7825-0/minicluster-data/master-2/wal/instance
I20260808 18:30:08.482975 7825 external_mini_cluster.cc:1428] Setting key 3798a34007047e259af0194fa789a537
I20260808 18:30:08.485752 27644 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:30:08 dist-test-slave-jztv krb5kdc[27324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.125@KRBTEST.COM for kudu/127.7.164.124@KRBTEST.COM
Aug 08 18:30:08 dist-test-slave-jztv krb5kdc[27324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.124@KRBTEST.COM
Aug 08 18:30:08 dist-test-slave-jztv krb5kdc[27324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.124@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:08.495783 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.480525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39668)
0808 18:30:08.480704 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.480728 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:08.480814 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:08.483972 (+ 3158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:08.483975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:08.484145 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:08.484238 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.484243 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.485929 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.485932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.486659 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.486671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.487599 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.487606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.487700 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.488361 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:08.488379 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:08.489941 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:08.493272 (+ 3331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.493279 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.493281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.493887 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.494298 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.494301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.494303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.494406 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.495201 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:08.495204 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:08.495316 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 18:30:08.495437 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 18:30:08.495594 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 18:30:08.497534 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.483649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39684 (local address 127.7.164.124:44815)
0808 18:30:08.484386 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.484395 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.484409 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:30:08.484444 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.484446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.484487 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.484579 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.485482 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.486318 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.487266 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.487696 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.487805 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.492457 (+ 4652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.492460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.492463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.492491 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.494112 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.494636 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.494639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.494640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.494725 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.495041 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.495048 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.495051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.495271 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.495427 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.496264 (+ 837us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.496691 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":646,"thread_start_us":94,"threads_started":1}
I20260808 18:30:08.497530 27648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.481283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39668 (local address 127.7.164.124:44815)
0808 18:30:08.481739 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.481754 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.481789 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:30:08.482673 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.482716 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.483122 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.483469 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.484394 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.485800 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.486789 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.487456 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.487640 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.490121 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.490127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.490136 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.490161 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.493139 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.494022 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.494028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.494036 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.494180 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.494504 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.494509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.494510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.495081 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.495217 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.496263 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.496617 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":323,"server-negotiator.queue_time_us":320,"thread_start_us":146,"threads_started":1}
I20260808 18:30:08.498885 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.487860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35068 (local address 127.7.164.126:41357)
0808 18:30:08.488002 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.488007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.488423 (+ 416us) server_negotiation.cc:407] Connection header received
0808 18:30:08.488565 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.488568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.488614 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.488681 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.489554 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.490306 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.491514 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.491813 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.491946 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.495326 (+ 3380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.495329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.495332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.495360 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.496956 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.497499 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.497504 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.497508 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.497603 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.497913 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.497919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.497923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.498134 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.498272 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.498608 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.498739 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:30:08.498905 27652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.487695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:41357 (local address 127.0.0.1:35068)
0808 18:30:08.488265 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.488319 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:08.488463 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:08.488763 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:08.488770 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:08.489285 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:08.489409 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.489415 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.490445 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.490452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.491385 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.491397 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.492055 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.492057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.492143 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.492653 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:08.492667 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:08.495146 (+ 2479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:08.497072 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.497080 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.497085 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.497350 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.497696 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.497699 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.497701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.497818 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.498267 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:08.498273 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:08.498400 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 18:30:08.498546 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 18:30:08.498718 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":109,"threads_started":1}
I20260808 18:30:08.500084 27644 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:30:08.502547 27537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167: Bootstrap starting.
I20260808 18:30:08.505534 27537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167: Neither blocks nor log segments found. Creating new log.
Aug 08 18:30:08 dist-test-slave-jztv krb5kdc[27324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.124@KRBTEST.COM for kudu/127.7.164.125@KRBTEST.COM
I20260808 18:30:08.506500 27537 log.cc:824] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:30:08 dist-test-slave-jztv krb5kdc[27324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:08.508983 27537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167: No bootstrap required, opened a new log
I20260808 18:30:08.510769 27652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.500604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:40863 (local address 127.0.0.1:45100)
0808 18:30:08.500722 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.500736 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:08.500804 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:08.502540 (+ 1736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:08.502543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:08.502717 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:08.502875 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.502889 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.503654 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.503659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.504189 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.504197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.504838 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.504840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.504925 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.505368 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:08.505382 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:08.506749 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:08.508975 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.508980 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.508983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.509230 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.509568 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.509574 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.509577 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.509684 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.510177 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:08.510180 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:08.510333 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 18:30:08.510454 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 18:30:08.510597 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 18:30:08.510767 27542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.500854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45100 (local address 127.7.164.125:40863)
0808 18:30:08.502348 (+ 1494us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.502352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.502364 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:30:08.502396 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.502398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.502432 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.502523 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.503045 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.503543 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.504312 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.504598 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.504707 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.506923 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.506929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.506935 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.506967 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.508873 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.509346 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.509358 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.509360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.509459 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.509801 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.509804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.509805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.510009 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.510128 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.510513 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.510628 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1415}
I20260808 18:30:08.511559 27644 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:30:08.512108 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.500856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35070 (local address 127.7.164.126:41357)
0808 18:30:08.500965 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.500969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.500982 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:30:08.501014 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.501016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.501054 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.501131 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.501780 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.502553 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.503609 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.503958 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.504037 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.507940 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.507944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.507946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.507973 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.509620 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.510098 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.510103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.510104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.510192 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.510515 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.510520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.510522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.510748 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.511335 (+ 587us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.511815 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.511951 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 18:30:08.513422 27537 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } }
I20260808 18:30:08.513692 27537 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:08.513751 27537 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1452adbd170486eb4d8cd5a0da42167, State: Initialized, Role: FOLLOWER
I20260808 18:30:08.514315 27537 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } }
I20260808 18:30:08.515563 27537 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:30:08.515710 27655 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } } }
I20260808 18:30:08.515856 27655 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 18:30:08 dist-test-slave-jztv krb5kdc[27324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.124@KRBTEST.COM for kudu/127.7.164.124@KRBTEST.COM
Aug 08 18:30:08 dist-test-slave-jztv krb5kdc[27324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.125@KRBTEST.COM
I20260808 18:30:08.520769 27652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.512070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39690)
0808 18:30:08.512280 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.512294 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:08.512365 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:08.512850 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:08.512853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:08.513026 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:08.513128 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.513138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.514070 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.514073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.514665 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.514677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.515243 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.515246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.515315 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.515775 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:08.515789 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:08.517302 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:08.519165 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.519168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.519169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.519429 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.519693 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.519696 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.519697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.519769 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.520286 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:08.520289 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:08.520376 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 18:30:08.520480 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 18:30:08.520611 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
W20260808 18:30:08.521277 27672 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:30:08.521348 27672 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:30:08.521445 27648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.512475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39690 (local address 127.7.164.124:44815)
0808 18:30:08.512589 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.512593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.512605 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:30:08.512635 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.512640 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.512686 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.512754 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.513253 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.513946 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.514791 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.515138 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.515513 (+ 375us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.517445 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.517451 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.517455 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.517478 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.519051 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.519533 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.519536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.519537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.519609 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.519847 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.519850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.519852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.520110 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.520207 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.521182 (+ 975us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.521299 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:30:08.519569 27442 master_runner.cc:186] Error getting master registration for 127.7.164.126:41357: OK, instance_id { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" instance_seqno: 1786213807995336 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:30:08.524253 27442 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:30:08.524544 27644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9: Bootstrap starting.
I20260808 18:30:08.524842 27542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.513262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45114 (local address 127.7.164.125:40863)
0808 18:30:08.513364 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.513378 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.513409 (+ 31us) server_negotiation.cc:407] Connection header received
0808 18:30:08.513544 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.513547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.513589 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.513665 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.514703 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.515429 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.516831 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.517132 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.517247 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.520727 (+ 3480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.520731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.520734 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.520760 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.522584 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.523087 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.523091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.523093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.523178 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.523493 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.523499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.523501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.523702 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.523863 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.524558 (+ 695us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.524712 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:30:08.527423 27644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:08.528194 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:30:08.528328 27644 log.cc:824] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:08.530781 27644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9: No bootstrap required, opened a new log
I20260808 18:30:08.535291 27644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } }
I20260808 18:30:08.535596 27644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:08.535667 27644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c6556f59abd4f009bfb14288acd1ca9, State: Initialized, Role: FOLLOWER
I20260808 18:30:08.536242 27644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } }
I20260808 18:30:08.537791 27644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:30:08.541456 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.530868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35082 (local address 127.7.164.126:41357)
0808 18:30:08.531002 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.531006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.531934 (+ 928us) server_negotiation.cc:407] Connection header received
0808 18:30:08.532111 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.532114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.532169 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.532225 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.533402 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.534124 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.535219 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.535530 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.535629 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.537455 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.537460 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.537462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.537492 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.539423 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.539965 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.539969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.539970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.540067 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.540391 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.540397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.540399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.540630 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.540740 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.541163 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.541295 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 18:30:08.543503 27703 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:30:08.543591 27703 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:30:08.541824 27544 master_runner.cc:186] Error getting master registration for 127.7.164.126:41357: OK, instance_id { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" instance_seqno: 1786213807995336 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:30:08.544879 27544 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:30:08.546849 27542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.530497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45122 (local address 127.7.164.125:40863)
0808 18:30:08.530613 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.530617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.531611 (+ 994us) server_negotiation.cc:407] Connection header received
0808 18:30:08.531733 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.531736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.531788 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.531862 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.532864 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.533646 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.534769 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.535095 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.535181 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.537469 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.537473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.537475 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.537515 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.539900 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.545311 (+ 5411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.545315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.545317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.545437 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.545788 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.545792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.545794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.546031 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.546156 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.546615 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.546735 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:30:08.550436 27689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } } }
I20260808 18:30:08.550612 27689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:30:08.551283 27648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.531301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39700 (local address 127.7.164.124:44815)
0808 18:30:08.531417 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.531421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.533265 (+ 1844us) server_negotiation.cc:407] Connection header received
0808 18:30:08.533358 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.533361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.533409 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.533495 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.537824 (+ 4329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.538598 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.539867 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.540210 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.540305 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.547335 (+ 7030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.547339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.547342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.547368 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.549289 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.549716 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.549720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.549721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.549808 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.550097 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.550101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.550103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.550369 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.550506 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.550981 (+ 475us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.551105 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 18:30:08.628190 27689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:30:08.628518 27689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } }
I20260808 18:30:08.630249 27504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c6556f59abd4f009bfb14288acd1ca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1452adbd170486eb4d8cd5a0da42167" is_pre_election: true
I20260808 18:30:08.630604 27504 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c6556f59abd4f009bfb14288acd1ca9 in term 0.
I20260808 18:30:08.630853 27689 leader_election.cc:288] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0ca2f941c64a74afe102fb48681f81 (127.7.164.126:41357), b1452adbd170486eb4d8cd5a0da42167 (127.7.164.125:40863)
I20260808 18:30:08.631031 27564 leader_election.cc:302] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c6556f59abd4f009bfb14288acd1ca9, b1452adbd170486eb4d8cd5a0da42167; no voters:
I20260808 18:30:08.631346 27689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:30:08.631438 27689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:30:08.631480 27689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:08.633387 27689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } }
I20260808 18:30:08.633461 27432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } attempt: 4
I20260808 18:30:08.634258 27689 leader_election.cc:288] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0ca2f941c64a74afe102fb48681f81 (127.7.164.126:41357), b1452adbd170486eb4d8cd5a0da42167 (127.7.164.125:40863)
I20260808 18:30:08.635761 27504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c6556f59abd4f009bfb14288acd1ca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1452adbd170486eb4d8cd5a0da42167"
I20260808 18:30:08.635893 27504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:08.637324 27504 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c6556f59abd4f009bfb14288acd1ca9 in term 1.
I20260808 18:30:08.637652 27564 leader_election.cc:302] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c6556f59abd4f009bfb14288acd1ca9, b1452adbd170486eb4d8cd5a0da42167; no voters:
I20260808 18:30:08.638985 27689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:30:08.639219 27689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [term 1 LEADER]: Becoming Leader. State: Replica: 7c6556f59abd4f009bfb14288acd1ca9, State: Running, Role: LEADER
I20260808 18:30:08.639611 27689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } }
I20260808 18:30:08.641573 27689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c6556f59abd4f009bfb14288acd1ca9. Latest consensus state: current_term: 1 leader_uuid: "7c6556f59abd4f009bfb14288acd1ca9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } } }
I20260808 18:30:08.641698 27689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:08.642096 27707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 18:30:08 dist-test-slave-jztv krb5kdc[27324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213807, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for kudu/127.7.164.124@KRBTEST.COM
I20260808 18:30:08.643182 27707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:30:08.649298 27504 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [term 1 FOLLOWER]: Refusing update from remote peer 7c6556f59abd4f009bfb14288acd1ca9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:30:08.649346 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.633944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44815 (local address 127.0.0.1:39708)
0808 18:30:08.634239 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:08.634255 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:08.634345 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:08.635059 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:08.635062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:08.635284 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:08.635401 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.635411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.638953 (+ 3542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.638957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.639778 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:08.639790 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.640841 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.640846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:08.640931 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.641600 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:08.641616 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:08.643300 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:08.647018 (+ 3718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.647022 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.647024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.647278 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.647831 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:08.647834 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:08.647836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:08.647942 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:08.648808 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:08.648811 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:08.648921 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 18:30:08.649019 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 18:30:08.649172 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185}
I20260808 18:30:08.650161 27705 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:30:08.650993 27648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:08.634569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39708 (local address 127.7.164.124:44815)
0808 18:30:08.634698 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:08.634704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:08.634718 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:30:08.634756 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:08.634762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:08.634813 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:08.634884 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:08.638006 (+ 3122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.638841 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.640074 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:08.640656 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:08.643544 (+ 2888us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:08.643794 (+ 250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:08.643799 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:08.643802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:08.643851 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:08.646846 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.647462 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.647465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.647466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.647647 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:08.648133 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:08.648136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:08.648138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:08.648616 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:08.650085 (+ 1469us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:08.650263 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 18:30:08.650668 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 18:30:08.654452 27432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81: Bootstrap starting.
I20260808 18:30:08.655375 27655 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c6556f59abd4f009bfb14288acd1ca9. Latest consensus state: current_term: 1 leader_uuid: "7c6556f59abd4f009bfb14288acd1ca9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } } }
I20260808 18:30:08.655517 27655 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:30:08.658052 27708 mvcc.cc:205] Tried to move back new op lower bound from 7316331760219795456 to 7316331760191201280. Current Snapshot: MvccSnapshot[applied={T|T < 7316331760219795456}]
I20260808 18:30:08.658524 27432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:08.658790 27705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c6556f59abd4f009bfb14288acd1ca9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } } }
I20260808 18:30:08.658898 27705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:08.659853 27432 log.cc:824] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:08.660085 27655 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c6556f59abd4f009bfb14288acd1ca9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } } }
I20260808 18:30:08.660189 27655 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:30:08.662792 27707 catalog_manager.cc:1347] Generated new cluster ID: b808b5930b644e258136cb4306d99e7a
I20260808 18:30:08.662859 27707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:30:08.672283 27709 mvcc.cc:205] Tried to move back new op lower bound from 7316331760219795456 to 7316331760191201280. Current Snapshot: MvccSnapshot[applied={T|T < 7316331760219795456}]
I20260808 18:30:08.677950 27432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81: No bootstrap required, opened a new log
I20260808 18:30:08.682349 27432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } }
I20260808 18:30:08.682723 27432 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:08.682829 27432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0ca2f941c64a74afe102fb48681f81, State: Initialized, Role: FOLLOWER
I20260808 18:30:08.683481 27432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } }
I20260808 18:30:08.685082 27432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:30:08.685369 27393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:08.687111 27393 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81 [term 1 FOLLOWER]: Refusing update from remote peer 7c6556f59abd4f009bfb14288acd1ca9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:30:08.688058 27689 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:30:08.687736 27392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c6556f59abd4f009bfb14288acd1ca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0ca2f941c64a74afe102fb48681f81" is_pre_election: true
I20260808 18:30:08.690433 27717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } } }
I20260808 18:30:08.693531 27717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 18:30:08.695115 27732 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:30:08.695433 27732 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:30:08.688241 27394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c6556f59abd4f009bfb14288acd1ca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0ca2f941c64a74afe102fb48681f81"
I20260808 18:30:08.697616 27717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c6556f59abd4f009bfb14288acd1ca9. Latest consensus state: current_term: 1 leader_uuid: "7c6556f59abd4f009bfb14288acd1ca9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } } }
I20260808 18:30:08.697777 27717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:30:08.697885 27717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c6556f59abd4f009bfb14288acd1ca9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } } }
I20260808 18:30:08.697988 27717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:30:08.705511 27736 mvcc.cc:205] Tried to move back new op lower bound from 7316331760219795456 to 7316331760191201280. Current Snapshot: MvccSnapshot[applied={T|T < 7316331760219795456}]
I20260808 18:30:08.726100 27689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c6556f59abd4f009bfb14288acd1ca9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0ca2f941c64a74afe102fb48681f81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41357 } } peers { permanent_uuid: "b1452adbd170486eb4d8cd5a0da42167" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 40863 } } peers { permanent_uuid: "7c6556f59abd4f009bfb14288acd1ca9" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44815 } } }
I20260808 18:30:08.726300 27689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:08.729844 27707 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:30:08.731267 27707 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:30:08.745091 27707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c6556f59abd4f009bfb14288acd1ca9: Generated new TSK 0
I20260808 18:30:08.746150 27707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:30:09.522737 27672 catalog_manager.cc:1259] Loaded cluster ID: b808b5930b644e258136cb4306d99e7a
I20260808 18:30:09.522876 27672 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167: loading cluster ID for follower catalog manager: success
I20260808 18:30:09.524547 27672 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167: acquiring CA information for follower catalog manager: success
I20260808 18:30:09.525739 27672 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b1452adbd170486eb4d8cd5a0da42167: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:30:09.696938 27732 catalog_manager.cc:1259] Loaded cluster ID: b808b5930b644e258136cb4306d99e7a
I20260808 18:30:09.697033 27732 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81: loading cluster ID for follower catalog manager: success
I20260808 18:30:09.698873 27732 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81: acquiring CA information for follower catalog manager: success
I20260808 18:30:09.700028 27732 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0a0ca2f941c64a74afe102fb48681f81: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:30:09.813961 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 27342
I20260808 18:30:09.831164 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 27442
I20260808 18:30:09.844056 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 27544
2026-08-08T18:30:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3412 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:30:09 dist-test-slave-jztv krb5kdc[27746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:30:09 dist-test-slave-jztv krb5kdc[27746](info): set up 2 sockets
Aug 08 18:30:09 dist-test-slave-jztv krb5kdc[27746](info): commencing operation
krb5kdc: starting...
W20260808 18:30:11.085340 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:30:11 dist-test-slave-jztv krb5kdc[27746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:30:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:30:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:30:11.219190 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:33201
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:33201
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:11.345129 27765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:11.345461 27765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:11.345543 27765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:11.349768 27765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:30:11.349866 27765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:11.349902 27765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:11.349944 27765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:30:11.349974 27765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:30:11.355366 27765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:33201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:33201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:11.356529 27765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:11.357553 27765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:11.366063 27775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:11.366118 27773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:11.366127 27778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:11.366322 27765 server_base.cc:1034] running on GCE node
I20260808 18:30:11.367004 27765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:11.368134 27765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:11.369329 27765 hybrid_clock.cc:648] HybridClock initialized: now 1786213811369315 us; error 30 us; skew 500 ppm
Aug 08 18:30:11 dist-test-slave-jztv krb5kdc[27746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213811, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:11.372144 27765 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:30:11.372978 27765 webserver.cc:460] Webserver started at http://127.7.164.126:38513/ using document root <none> and password file <none>
I20260808 18:30:11.373494 27765 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:11.373593 27765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:11.373766 27765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:30:11.375923 27765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "b9dde325b7b244e89806f960176900dd"
format_stamp: "Formatted at 2026-08-08 18:30:11 on dist-test-slave-jztv"
server_key: "ff55268ebdffcac55d545538aa53789d"
server_key_iv: "9e9ef5b13c4bfabbafb63de5f14113ba"
server_key_version: "encryptionkey@0"
I20260808 18:30:11.376452 27765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "b9dde325b7b244e89806f960176900dd"
format_stamp: "Formatted at 2026-08-08 18:30:11 on dist-test-slave-jztv"
server_key: "ff55268ebdffcac55d545538aa53789d"
server_key_iv: "9e9ef5b13c4bfabbafb63de5f14113ba"
server_key_version: "encryptionkey@0"
I20260808 18:30:11.380183 27765 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:30:11.382608 27794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:11.383879 27765 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:30:11.384012 27765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "b9dde325b7b244e89806f960176900dd"
format_stamp: "Formatted at 2026-08-08 18:30:11 on dist-test-slave-jztv"
server_key: "ff55268ebdffcac55d545538aa53789d"
server_key_iv: "9e9ef5b13c4bfabbafb63de5f14113ba"
server_key_version: "encryptionkey@0"
I20260808 18:30:11.384127 27765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:11.394613 27765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:11.398109 27765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:11.398401 27765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:11.405645 27765 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:33201
I20260808 18:30:11.405719 27861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:33201 every 8 connection(s)
I20260808 18:30:11.406715 27765 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:30:11.407495 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 27765
I20260808 18:30:11.407630 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:30:11.407923 7825 external_mini_cluster.cc:1428] Setting key d57f0ca497d5e0ef777e7f12807952b7
I20260808 18:30:11.410122 27862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:11.417027 27862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd: Bootstrap starting.
Aug 08 18:30:11 dist-test-slave-jztv krb5kdc[27746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:11.419471 27862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:11.420219 27862 log.cc:824] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:11.422310 27862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd: No bootstrap required, opened a new log
I20260808 18:30:11.422894 27868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:11.409358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55316 (local address 127.7.164.126:33201)
0808 18:30:11.409802 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:11.409827 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:11.409862 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:30:11.410603 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:11.410624 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:11.410953 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:11.411250 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:11.412200 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:11.413205 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:11.414424 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:11.414900 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:11.415029 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:11.417949 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:11.417955 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:11.417968 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:11.417996 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:11.419886 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:11.420368 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:11.420374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:11.420379 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:11.420491 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:11.420803 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:11.420808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:11.420810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:11.421218 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:11.421438 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:11.421736 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 18:30:11.422034 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":171,"threads_started":1}
I20260808 18:30:11.425748 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:30:11.426294 27862 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dde325b7b244e89806f960176900dd" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33201 } }
I20260808 18:30:11.426553 27862 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:11.426609 27862 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9dde325b7b244e89806f960176900dd, State: Initialized, Role: FOLLOWER
I20260808 18:30:11.427049 27862 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dde325b7b244e89806f960176900dd" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33201 } }
I20260808 18:30:11.427181 27862 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:30:11.427265 27862 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:30:11.427407 27862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:11.428323 27862 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dde325b7b244e89806f960176900dd" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33201 } }
I20260808 18:30:11.428629 27862 leader_election.cc:302] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9dde325b7b244e89806f960176900dd; no voters:
I20260808 18:30:11.428889 27862 leader_election.cc:288] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:30:11.429052 27871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:30:11.429317 27871 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [term 1 LEADER]: Becoming Leader. State: Replica: b9dde325b7b244e89806f960176900dd, State: Running, Role: LEADER
I20260808 18:30:11.429627 27871 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dde325b7b244e89806f960176900dd" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33201 } }
I20260808 18:30:11.430284 27862 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:30:11.431603 27873 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9dde325b7b244e89806f960176900dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dde325b7b244e89806f960176900dd" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33201 } } }
I20260808 18:30:11.431773 27873 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:11.432217 27887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:30:11.432057 27874 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9dde325b7b244e89806f960176900dd. Latest consensus state: current_term: 1 leader_uuid: "b9dde325b7b244e89806f960176900dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9dde325b7b244e89806f960176900dd" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33201 } } }
I20260808 18:30:11.432761 27874 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:11.435307 27887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:30:11.440900 27887 catalog_manager.cc:1347] Generated new cluster ID: 7eca973f771940c5b75d50e59d3fcca5
I20260808 18:30:11.440989 27887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:30:11.483609 27887 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:30:11.484644 27887 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:30:11.489518 27887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9dde325b7b244e89806f960176900dd: Generated new TSK 0
I20260808 18:30:11.490272 27887 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:30:11.533720 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 27765
2026-08-08T18:30:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1685 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:30:11 dist-test-slave-jztv krb5kdc[27902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:30:11 dist-test-slave-jztv krb5kdc[27902](info): set up 2 sockets
Aug 08 18:30:11 dist-test-slave-jztv krb5kdc[27902](info): commencing operation
krb5kdc: starting...
W20260808 18:30:12.768540 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:30:12 dist-test-slave-jztv krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:30:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:30:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:30:12.901932 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:45267
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:45267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:13.016788 27920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:13.017094 27920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:13.017179 27920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:13.021263 27920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:30:13.021361 27920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:13.021397 27920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:13.021438 27920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:30:13.021469 27920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:30:13.027115 27920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:45267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:45267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:13.028345 27920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:13.029470 27920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:30:13.038268 27920 server_base.cc:1034] running on GCE node
W20260808 18:30:13.038293 27928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:13.038331 27925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:13.038271 27926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:13.039103 27920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:13.040166 27920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:13.041358 27920 hybrid_clock.cc:648] HybridClock initialized: now 1786213813041343 us; error 28 us; skew 500 ppm
Aug 08 18:30:13 dist-test-slave-jztv krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213813, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:13.044262 27920 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:30:13.045142 27920 webserver.cc:460] Webserver started at http://127.7.164.126:36539/ using document root <none> and password file <none>
I20260808 18:30:13.045708 27920 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:13.045804 27920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:13.046053 27920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:30:13.048137 27920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "8962b17c345d40fd9c9c39e1b5d70f9a"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "271471d1d91d7e241e6f3f2dc79e15d5"
server_key_iv: "beb9974d4c9ea8bbdc619a5255ffad63"
server_key_version: "encryptionkey@0"
I20260808 18:30:13.048719 27920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "8962b17c345d40fd9c9c39e1b5d70f9a"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "271471d1d91d7e241e6f3f2dc79e15d5"
server_key_iv: "beb9974d4c9ea8bbdc619a5255ffad63"
server_key_version: "encryptionkey@0"
I20260808 18:30:13.052372 27920 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:30:13.054759 27939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:13.055991 27920 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:30:13.056151 27920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "8962b17c345d40fd9c9c39e1b5d70f9a"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "271471d1d91d7e241e6f3f2dc79e15d5"
server_key_iv: "beb9974d4c9ea8bbdc619a5255ffad63"
server_key_version: "encryptionkey@0"
I20260808 18:30:13.056268 27920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:13.068687 27920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:13.072049 27920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:13.072264 27920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:13.079397 27920 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:45267
I20260808 18:30:13.079793 28023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:45267 every 8 connection(s)
I20260808 18:30:13.080533 27920 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:30:13.083371 28024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:13.088721 28024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Bootstrap starting.
I20260808 18:30:13.089798 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 27920
I20260808 18:30:13.089960 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:30:13.090292 7825 external_mini_cluster.cc:1428] Setting key 0d3e5bfbf337540e34451507edb43fff
I20260808 18:30:13.091688 28024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:13.092443 28024 log.cc:824] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:13.094477 28024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: No bootstrap required, opened a new log
I20260808 18:30:13.097924 28024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } }
I20260808 18:30:13.098191 28024 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:13.098296 28024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8962b17c345d40fd9c9c39e1b5d70f9a, State: Initialized, Role: FOLLOWER
I20260808 18:30:13.098826 28024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } }
I20260808 18:30:13.099020 28024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:30:13.099078 28024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:30:13.099182 28024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 0 FOLLOWER]: Advancing to term 1
Aug 08 18:30:13 dist-test-slave-jztv krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:13.100240 28024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } }
I20260808 18:30:13.100595 28024 leader_election.cc:302] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8962b17c345d40fd9c9c39e1b5d70f9a; no voters:
I20260808 18:30:13.100965 28024 leader_election.cc:288] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:30:13.101033 28030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:30:13.101240 28030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 1 LEADER]: Becoming Leader. State: Replica: 8962b17c345d40fd9c9c39e1b5d70f9a, State: Running, Role: LEADER
I20260808 18:30:13.101608 28030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } }
I20260808 18:30:13.102362 28024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:30:13.102989 28030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8962b17c345d40fd9c9c39e1b5d70f9a. Latest consensus state: current_term: 1 leader_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } } }
I20260808 18:30:13.103190 28030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:13.103530 28031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } } }
I20260808 18:30:13.103680 28031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:13.103768 28040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:30:13.106977 28027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:13.091482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39734 (local address 127.7.164.126:45267)
0808 18:30:13.091985 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:13.092001 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:13.092036 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:30:13.092816 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:13.092843 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:13.093258 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:13.093637 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:13.094530 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.095568 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.096448 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.096941 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.097162 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:13.100240 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:13.100246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:13.100260 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:13.100291 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:13.102936 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:13.103655 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:13.103663 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:13.103671 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:13.103887 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:13.104292 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:13.104296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:13.104298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:13.104909 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:13.105137 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:13.105480 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 18:30:13.105855 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":144,"threads_started":1}
I20260808 18:30:13.107309 28040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:30:13.114110 28040 catalog_manager.cc:1347] Generated new cluster ID: 9b8a07200a6d40ad8672f352fbcab626
I20260808 18:30:13.114207 28040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:30:13.144392 28040 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:30:13.145422 28040 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:30:13.149924 28040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Generated new TSK 0
I20260808 18:30:13.150805 28040 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:30:13.197829 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45267
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:13.341385 28054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:13.341687 28054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:13.341763 28054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:13.345920 28054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:13.346024 28054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:13.346123 28054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:30:13.351497 28054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:13.352532 28054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:13.353551 28054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:13.363265 28062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:13.363255 28064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:13.363237 28061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:13.363686 28054 server_base.cc:1034] running on GCE node
I20260808 18:30:13.364136 28054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:13.364768 28054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:13.365962 28054 hybrid_clock.cc:648] HybridClock initialized: now 1786213813365950 us; error 32 us; skew 500 ppm
Aug 08 18:30:13 dist-test-slave-jztv krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213813, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:13.368820 28054 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:30:13.369751 28054 webserver.cc:460] Webserver started at http://127.7.164.65:43107/ using document root <none> and password file <none>
I20260808 18:30:13.370414 28054 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:13.370512 28054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:13.370725 28054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:30:13.372752 28054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "c27b291221ae41158d078f84c3e2811e"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "732405baeac4196a708b3d57143b397c"
server_key_iv: "c8fd6a91464c6ebb7e2aec4188470df3"
server_key_version: "encryptionkey@0"
I20260808 18:30:13.373337 28054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "c27b291221ae41158d078f84c3e2811e"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "732405baeac4196a708b3d57143b397c"
server_key_iv: "c8fd6a91464c6ebb7e2aec4188470df3"
server_key_version: "encryptionkey@0"
I20260808 18:30:13.377136 28054 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:30:13.379616 28075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:13.381008 28054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:30:13.381109 28054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "c27b291221ae41158d078f84c3e2811e"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "732405baeac4196a708b3d57143b397c"
server_key_iv: "c8fd6a91464c6ebb7e2aec4188470df3"
server_key_version: "encryptionkey@0"
I20260808 18:30:13.381196 28054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:13.399914 28054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:13.403609 28054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:13.403849 28054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:13.404562 28054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:13.405439 28054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:13.405489 28054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:13.405560 28054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:13.405592 28054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:13.414824 28054 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:46853
I20260808 18:30:13.414902 28241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:46853 every 8 connection(s)
I20260808 18:30:13.415774 28054 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:30:13.417093 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 28054
I20260808 18:30:13.417222 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:30:13.417537 7825 external_mini_cluster.cc:1428] Setting key 590e2f90c0ee33405aa1177d3e111356
Aug 08 18:30:13 dist-test-slave-jztv krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213813, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:30:13.430652 28027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:13.417931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:54923 (local address 127.7.164.126:45267)
0808 18:30:13.418093 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:13.418098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:13.419434 (+ 1336us) server_negotiation.cc:407] Connection header received
0808 18:30:13.420763 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:13.420766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:13.420812 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:13.420885 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:13.422301 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.422846 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.424028 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.424278 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.424389 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:13.426977 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:13.426981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:13.426985 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:13.427015 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:13.428476 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:13.428991 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:13.428994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:13.428996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:13.429096 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:13.429455 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:13.429458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:13.429460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:13.429642 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:13.429747 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:13.430335 (+ 588us) server_negotiation.cc:299] Negotiation successful
0808 18:30:13.430477 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
I20260808 18:30:13.431531 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:13.418447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:54923)
0808 18:30:13.419255 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:13.419314 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:13.420541 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:13.421073 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:13.421084 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:13.421627 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:13.422120 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:13.422134 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.422986 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.422990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:13.423890 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:13.423899 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.424431 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.424436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:13.424631 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:13.425279 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:13.425296 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:13.426796 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:13.428592 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:13.428598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:13.428613 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:13.428867 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:13.429197 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:13.429200 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:13.429202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:13.429343 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:13.429758 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:13.429766 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:13.430013 (+ 247us) client_negotiation.cc:769] Sending connection context
0808 18:30:13.430265 (+ 252us) client_negotiation.cc:240] Negotiation successful
0808 18:30:13.430632 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":130,"threads_started":1}
I20260808 18:30:13.432796 28242 heartbeater.cc:347] Connected to a master server at 127.7.164.126:45267
I20260808 18:30:13.433054 28242 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:13.433555 28242 heartbeater.cc:511] Master 127.7.164.126:45267 requested a full tablet report, sending...
I20260808 18:30:13.435057 27965 ts_manager.cc:194] Registered new tserver with Master: c27b291221ae41158d078f84c3e2811e (127.7.164.65:46853)
I20260808 18:30:13.436209 27965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:54923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:30:13.447507 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45267
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:13.563125 28250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:13.563416 28250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:13.563493 28250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:13.567587 28250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:13.567682 28250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:13.567775 28250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:30:13.573232 28250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:13.574405 28250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:13.575455 28250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:13.585603 28257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:13.585705 28250 server_base.cc:1034] running on GCE node
W20260808 18:30:13.585615 28261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:13.585605 28255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:13.586347 28250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:13.587071 28250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:13.588264 28250 hybrid_clock.cc:648] HybridClock initialized: now 1786213813588245 us; error 46 us; skew 500 ppm
Aug 08 18:30:13 dist-test-slave-jztv krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213813, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:13.591559 28250 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:30:13.592675 28250 webserver.cc:460] Webserver started at http://127.7.164.66:40615/ using document root <none> and password file <none>
I20260808 18:30:13.593334 28250 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:13.593444 28250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:13.593716 28250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:30:13.596133 28250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "bbe65ce30d3144c0a865e4f2bfa4162c"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "328fa7c30a4758b6fce308172c983704"
server_key_iv: "fa58761314c44a094881a986e5a78304"
server_key_version: "encryptionkey@0"
I20260808 18:30:13.596793 28250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "bbe65ce30d3144c0a865e4f2bfa4162c"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "328fa7c30a4758b6fce308172c983704"
server_key_iv: "fa58761314c44a094881a986e5a78304"
server_key_version: "encryptionkey@0"
I20260808 18:30:13.601168 28250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:30:13.604312 28271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:13.606107 28250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:30:13.606299 28250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "bbe65ce30d3144c0a865e4f2bfa4162c"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "328fa7c30a4758b6fce308172c983704"
server_key_iv: "fa58761314c44a094881a986e5a78304"
server_key_version: "encryptionkey@0"
I20260808 18:30:13.606420 28250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:13.631518 28250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:13.635610 28250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:13.635826 28250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:13.636511 28250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:13.637436 28250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:13.637485 28250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:13.637552 28250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:13.637588 28250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:13.647127 28250 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:41509
I20260808 18:30:13.647217 28426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:41509 every 8 connection(s)
I20260808 18:30:13.648140 28250 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:30:13.655556 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 28250
I20260808 18:30:13.655704 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:30:13.655995 7825 external_mini_cluster.cc:1428] Setting key 18a58de9206d729cd6c9223d06b21d2e
Aug 08 18:30:13 dist-test-slave-jztv krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213813, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:13.663869 28027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:13.650202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:38477 (local address 127.7.164.126:45267)
0808 18:30:13.650395 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:13.650401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:13.651422 (+ 1021us) server_negotiation.cc:407] Connection header received
0808 18:30:13.652813 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:13.652816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:13.652867 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:13.652950 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:13.654269 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.654876 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.656229 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.656539 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.656631 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:13.659865 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:13.659872 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:13.659878 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:13.659917 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:13.661509 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:13.662079 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:13.662083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:13.662085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:13.662174 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:13.662600 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:13.662604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:13.662606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:13.662811 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:13.662927 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:13.663554 (+ 627us) server_negotiation.cc:299] Negotiation successful
0808 18:30:13.663696 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 18:30:13.664803 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:13.650752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:38477)
0808 18:30:13.651252 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:13.651299 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:13.652581 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:13.653130 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:13.653137 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:13.653535 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:13.654046 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:13.654060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.655017 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.655021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:13.656069 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:13.656100 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.656891 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.656894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:13.657062 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:13.657807 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:13.657830 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:13.659648 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:13.661644 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:13.661650 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:13.661668 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:13.661956 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:13.662314 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:13.662317 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:13.662319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:13.662480 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:13.662926 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:13.662935 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:13.663225 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 18:30:13.663495 (+ 270us) client_negotiation.cc:240] Negotiation successful
0808 18:30:13.663882 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":135,"threads_started":1}
I20260808 18:30:13.666304 28427 heartbeater.cc:347] Connected to a master server at 127.7.164.126:45267
I20260808 18:30:13.666603 28427 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:13.667160 28427 heartbeater.cc:511] Master 127.7.164.126:45267 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
I20260808 18:30:13.668411 27965 ts_manager.cc:194] Registered new tserver with Master: bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509)
I20260808 18:30:13.669102 27965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:38477
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:30:13.689543 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45267
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:13.807991 28435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:13.808290 28435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:13.808393 28435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:13.813246 28435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:13.813311 28435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:13.813443 28435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:30:13.819222 28435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:13.820376 28435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:13.821439 28435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:13.831596 28443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:13.831601 28449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:13.831666 28435 server_base.cc:1034] running on GCE node
W20260808 18:30:13.831630 28447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:13.832262 28435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:13.832988 28435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:13.834251 28435 hybrid_clock.cc:648] HybridClock initialized: now 1786213813834176 us; error 53 us; skew 500 ppm
Aug 08 18:30:13 dist-test-slave-jztv krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213813, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:13.837234 28435 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:30:13.838307 28435 webserver.cc:460] Webserver started at http://127.7.164.67:35047/ using document root <none> and password file <none>
I20260808 18:30:13.838910 28435 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:13.839010 28435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:13.839257 28435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:30:13.841358 28435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "d0d96fd4d43c4b25b304d804885f6bee"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "60052cddb8e7954e48c110504325204b"
server_key_iv: "9c37dfe00b6b257db78ddfcdc2d36d07"
server_key_version: "encryptionkey@0"
I20260808 18:30:13.841960 28435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "d0d96fd4d43c4b25b304d804885f6bee"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "60052cddb8e7954e48c110504325204b"
server_key_iv: "9c37dfe00b6b257db78ddfcdc2d36d07"
server_key_version: "encryptionkey@0"
I20260808 18:30:13.845706 28435 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:30:13.848248 28458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:13.849527 28435 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:30:13.849680 28435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "d0d96fd4d43c4b25b304d804885f6bee"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "60052cddb8e7954e48c110504325204b"
server_key_iv: "9c37dfe00b6b257db78ddfcdc2d36d07"
server_key_version: "encryptionkey@0"
I20260808 18:30:13.849800 28435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:13.866485 28435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:13.869676 28435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:13.869910 28435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:13.870604 28435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:13.871515 28435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:13.871590 28435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:13.871667 28435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:13.871718 28435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:13.881044 28435 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:39835
I20260808 18:30:13.881075 28610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:39835 every 8 connection(s)
I20260808 18:30:13.882059 28435 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:30:13.887110 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 28435
I20260808 18:30:13.887253 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:30:13.887538 7825 external_mini_cluster.cc:1428] Setting key 4a2f06f792cdbf6462eb3a7a690f0a61
Aug 08 18:30:13 dist-test-slave-jztv krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213813, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:13.897156 28027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:13.884063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:48345 (local address 127.7.164.126:45267)
0808 18:30:13.884213 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:13.884217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:13.885421 (+ 1204us) server_negotiation.cc:407] Connection header received
0808 18:30:13.886464 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:13.886468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:13.886530 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:13.886638 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:13.888102 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.888886 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.890473 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.890704 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.890819 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:13.893390 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:13.893399 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:13.893402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:13.893428 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:13.894995 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:13.895561 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:13.895569 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:13.895570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:13.895649 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:13.896000 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:13.896005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:13.896007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:13.896202 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:13.896316 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:13.896838 (+ 522us) server_negotiation.cc:299] Negotiation successful
0808 18:30:13.896980 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:30:13.897976 28614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:13.884512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:48345)
0808 18:30:13.885251 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:13.885300 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:13.886199 (+ 899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:13.886846 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:13.886853 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:13.887263 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:13.887914 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:13.887937 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.889043 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.889047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:13.890336 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:13.890346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.890864 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.890868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:13.891055 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:13.891732 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:13.891749 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:13.893216 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:13.895123 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:13.895130 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:13.895147 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:13.895442 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:13.895743 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:13.895747 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:13.895749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:13.895898 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:13.896316 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:13.896324 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:13.896588 (+ 264us) client_negotiation.cc:769] Sending connection context
0808 18:30:13.896793 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 18:30:13.897136 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":117,"threads_started":1}
I20260808 18:30:13.899281 28611 heartbeater.cc:347] Connected to a master server at 127.7.164.126:45267
I20260808 18:30:13.899544 28611 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:13.900024 28611 heartbeater.cc:511] Master 127.7.164.126:45267 requested a full tablet report, sending...
I20260808 18:30:13.901238 27965 ts_manager.cc:194] Registered new tserver with Master: d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835)
I20260808 18:30:13.901922 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:30:13.902002 27965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:48345
I20260808 18:30:13.912073 28027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:13.904273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39748 (local address 127.7.164.126:45267)
0808 18:30:13.904406 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:13.904411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:13.904684 (+ 273us) server_negotiation.cc:407] Connection header received
0808 18:30:13.904762 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:13.904765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:13.904804 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:13.904875 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:13.905563 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.906061 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.906920 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.907131 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.907211 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:13.908606 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:13.908609 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:13.908611 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:13.908633 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:13.909980 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:13.910564 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:13.910566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:13.910568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:13.910647 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:13.910946 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:13.910953 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:13.910955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:13.911201 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:13.911314 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:13.911705 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 18:30:13.911871 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 18:30:13.915405 27965 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:30:13.918423 27965 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:30:13.933419 28631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:13.928025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.7.164.65:46853)
0808 18:30:13.929042 (+ 1017us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:13.929058 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:13.929081 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:30:13.929150 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:13.929154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:13.929320 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:13.929474 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:13.929723 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.930827 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.931930 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.932709 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.932821 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:13.932977 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:13.933136 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 18:30:13.933237 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":95,"threads_started":1}
I20260808 18:30:13.933959 28630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:13.927843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:46853 (local address 127.0.0.1:34774)
0808 18:30:13.928496 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:13.928510 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:13.928598 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:13.929478 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:13.929481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:13.929501 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:13.929592 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:13.929598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.930965 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.930971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:13.931778 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:13.931788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.932833 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.932835 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:13.932915 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:13.932948 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 18:30:13.933086 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 18:30:13.933771 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":33,"thread_start_us":112,"threads_started":1}
I20260808 18:30:13.934522 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:13.928124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38298 (local address 127.7.164.66:41509)
0808 18:30:13.928445 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:13.928457 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:13.928479 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:30:13.928576 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:13.928582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:13.928773 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:13.928934 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:13.929196 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.930279 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.932386 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.933198 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.933300 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:13.933442 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:13.934263 (+ 821us) server_negotiation.cc:299] Negotiation successful
0808 18:30:13.934372 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":140,"threads_started":1}
I20260808 18:30:13.934638 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:13.927007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:39835 (local address 127.0.0.1:50368)
0808 18:30:13.927417 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:13.927472 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:13.927616 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:13.928704 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:13.928710 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:13.928734 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:13.928888 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:13.928896 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.930448 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.930452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:13.932074 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:13.932087 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.934016 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.934022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:13.934136 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:13.934161 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:30:13.934388 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 18:30:13.934470 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":155,"thread_start_us":136,"threads_started":1}
I20260808 18:30:13.934640 28628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:13.927694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:41509 (local address 127.0.0.1:38298)
0808 18:30:13.928177 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:13.928190 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:13.928285 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:13.928913 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:13.928916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:13.928934 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:13.929056 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:13.929066 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.930426 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.930432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:13.932074 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:13.932087 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.933508 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.933511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:13.934040 (+ 529us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:13.934063 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:30:13.934365 (+ 302us) client_negotiation.cc:240] Negotiation successful
0808 18:30:13.934448 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":510,"thread_start_us":99,"threads_started":1}
I20260808 18:30:13.935050 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:13.927178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50368 (local address 127.7.164.67:39835)
0808 18:30:13.928231 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:13.928245 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:13.928269 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:30:13.928350 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:13.928359 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:13.928546 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:13.928712 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:13.929025 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.930280 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.933057 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:13.933885 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:13.933995 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:13.934145 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:13.934804 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 18:30:13.934904 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":134,"threads_started":1}
I20260808 18:30:13.937325 28149 tablet_service.cc:1467] Processing CreateTablet for tablet 289c180b0a454ed69b9677104627d239 (DEFAULT_TABLE table=test-table [id=e61be8594f0f4cf198025ac2670a32ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:13.937493 28524 tablet_service.cc:1467] Processing CreateTablet for tablet 289c180b0a454ed69b9677104627d239 (DEFAULT_TABLE table=test-table [id=e61be8594f0f4cf198025ac2670a32ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:13.937351 28335 tablet_service.cc:1467] Processing CreateTablet for tablet 289c180b0a454ed69b9677104627d239 (DEFAULT_TABLE table=test-table [id=e61be8594f0f4cf198025ac2670a32ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:13.938488 28335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289c180b0a454ed69b9677104627d239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:13.938500 28149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289c180b0a454ed69b9677104627d239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:13.938508 28524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289c180b0a454ed69b9677104627d239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:13.944813 28636 tablet_bootstrap.cc:492] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Bootstrap starting.
I20260808 18:30:13.946141 28637 tablet_bootstrap.cc:492] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Bootstrap starting.
I20260808 18:30:13.946574 28638 tablet_bootstrap.cc:492] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee: Bootstrap starting.
I20260808 18:30:13.947597 28636 tablet_bootstrap.cc:654] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:13.948558 28636 log.cc:824] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:13.949308 28638 tablet_bootstrap.cc:654] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:13.950070 28637 tablet_bootstrap.cc:654] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:13.950134 28638 log.cc:824] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:13.950552 28636 tablet_bootstrap.cc:492] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: No bootstrap required, opened a new log
I20260808 18:30:13.950785 28636 ts_tablet_manager.cc:1397] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:30:13.950933 28637 log.cc:824] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:13.952171 28638 tablet_bootstrap.cc:492] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee: No bootstrap required, opened a new log
I20260808 18:30:13.952364 28638 ts_tablet_manager.cc:1397] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260808 18:30:13.952859 28637 tablet_bootstrap.cc:492] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: No bootstrap required, opened a new log
I20260808 18:30:13.953060 28637 ts_tablet_manager.cc:1397] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260808 18:30:13.954492 28636 raft_consensus.cc:348] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:13.954917 28636 raft_consensus.cc:374] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:13.955032 28636 raft_consensus.cc:729] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe65ce30d3144c0a865e4f2bfa4162c, State: Initialized, Role: FOLLOWER
I20260808 18:30:13.955643 28636 consensus_queue.cc:260] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:13.956338 28427 heartbeater.cc:503] Master 127.7.164.126:45267 was elected leader, sending a full tablet report...
I20260808 18:30:13.956458 28638 raft_consensus.cc:348] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:13.956738 28638 raft_consensus.cc:374] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:13.956835 28638 raft_consensus.cc:729] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0d96fd4d43c4b25b304d804885f6bee, State: Initialized, Role: FOLLOWER
I20260808 18:30:13.957299 28638 consensus_queue.cc:260] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:13.957619 28637 raft_consensus.cc:348] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:13.957824 28636 ts_tablet_manager.cc:1428] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 18:30:13.957927 28637 raft_consensus.cc:374] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:13.958012 28637 raft_consensus.cc:729] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c27b291221ae41158d078f84c3e2811e, State: Initialized, Role: FOLLOWER
I20260808 18:30:13.958381 28611 heartbeater.cc:503] Master 127.7.164.126:45267 was elected leader, sending a full tablet report...
I20260808 18:30:13.958480 28638 ts_tablet_manager.cc:1428] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260808 18:30:13.958465 28637 consensus_queue.cc:260] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:13.959080 28242 heartbeater.cc:503] Master 127.7.164.126:45267 was elected leader, sending a full tablet report...
I20260808 18:30:13.959470 28637 ts_tablet_manager.cc:1428] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:30:14.103343 28643 raft_consensus.cc:482] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:30:14.103595 28643 raft_consensus.cc:504] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:14.104912 28643 leader_election.cc:288] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835), c27b291221ae41158d078f84c3e2811e (127.7.164.65:46853)
I20260808 18:30:14.106814 28644 raft_consensus.cc:482] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:30:14.106992 28644 raft_consensus.cc:504] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:14.108141 28644 leader_election.cc:288] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c27b291221ae41158d078f84c3e2811e (127.7.164.65:46853), bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509)
I20260808 18:30:14.109964 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.105276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:39835 (local address 127.7.164.66:53259)
0808 18:30:14.105441 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:14.105461 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:14.105590 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:14.105851 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:14.105855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:14.105875 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:14.106032 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.106038 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.106969 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.106974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.108113 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.108124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.109487 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.109494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.109613 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.109640 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:30:14.109701 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 18:30:14.109793 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"spinlock_wait_cycles":39808}
I20260808 18:30:14.109968 28631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.105735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:41793 (local address 127.7.164.65:46853)
0808 18:30:14.105873 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:14.105879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:14.105894 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:30:14.106083 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:14.106087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:14.106135 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:14.106250 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:14.106523 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.107238 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.108687 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.109294 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.109396 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.109471 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:14.109794 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 18:30:14.109870 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:30:14.110438 28646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.105579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:46853 (local address 127.7.164.66:41793)
0808 18:30:14.105815 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:14.105828 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:14.105951 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:14.106261 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:14.106264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:14.106281 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:14.106410 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.106416 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.107362 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.107365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.108530 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.108544 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.109554 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.109557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.109653 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.109684 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 18:30:14.110192 (+ 508us) client_negotiation.cc:240] Negotiation successful
0808 18:30:14.110289 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"spinlock_wait_cycles":23936,"thread_start_us":79,"threads_started":1}
I20260808 18:30:14.110793 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.105418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:53259 (local address 127.7.164.67:39835)
0808 18:30:14.105580 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:14.105585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:14.105605 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:30:14.105693 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:14.105696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:14.105751 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:14.105840 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:14.106151 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.106850 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.108713 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.109305 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.109416 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.110561 (+ 1145us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:14.110600 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 18:30:14.110678 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1057,"server-negotiator.queue_time_us":63}
I20260808 18:30:14.111617 28548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289c180b0a454ed69b9677104627d239" candidate_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0d96fd4d43c4b25b304d804885f6bee" is_pre_election: true
I20260808 18:30:14.111827 28548 raft_consensus.cc:2454] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbe65ce30d3144c0a865e4f2bfa4162c in term 0.
I20260808 18:30:14.112334 28274 leader_election.cc:302] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbe65ce30d3144c0a865e4f2bfa4162c, d0d96fd4d43c4b25b304d804885f6bee; no voters:
I20260808 18:30:14.112628 28643 raft_consensus.cc:2781] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:30:14.112732 28643 raft_consensus.cc:482] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:30:14.112780 28643 raft_consensus.cc:3037] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:14.112885 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.108492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:34327 (local address 127.7.164.66:41509)
0808 18:30:14.108621 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:14.108625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:14.108899 (+ 274us) server_negotiation.cc:407] Connection header received
0808 18:30:14.109094 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:14.109098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:14.109145 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:14.109198 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:14.109856 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.110589 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.111761 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.112278 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.112405 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.112450 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:14.112695 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 18:30:14.112778 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:30:14.113058 28177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289c180b0a454ed69b9677104627d239" candidate_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c27b291221ae41158d078f84c3e2811e" is_pre_election: true
I20260808 18:30:14.113186 28614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.108266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:46853 (local address 127.7.164.67:37517)
0808 18:30:14.108441 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:14.108454 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:14.108533 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:14.109357 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:14.109360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:14.109416 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:14.109556 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.109564 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.110678 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.110681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.111896 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.111906 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.112779 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.112783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.112890 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.112913 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:30:14.112967 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 18:30:14.113029 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":78,"spinlock_wait_cycles":3328}
I20260808 18:30:14.113386 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.108412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:41509 (local address 127.7.164.67:34327)
0808 18:30:14.108810 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:14.108824 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:14.108970 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:14.109347 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:14.109350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:14.109367 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:14.109740 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.109747 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.110735 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.110739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.111630 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.111639 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.112402 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.112405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.112521 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.112560 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 18:30:14.112996 (+ 436us) client_negotiation.cc:240] Negotiation successful
0808 18:30:14.113084 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":41,"spinlock_wait_cycles":19712,"thread_start_us":138,"threads_started":1}
I20260808 18:30:14.113344 28177 raft_consensus.cc:2454] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbe65ce30d3144c0a865e4f2bfa4162c in term 0.
I20260808 18:30:14.113502 28649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.108751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:37517 (local address 127.7.164.65:46853)
0808 18:30:14.109124 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:14.109144 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:14.109156 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:30:14.109200 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:14.109202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:14.109244 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:14.109515 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:14.109684 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.110530 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.112041 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.112646 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.113180 (+ 534us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.113222 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:14.113257 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 18:30:14.113312 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":142,"threads_started":1}
I20260808 18:30:14.114195 28643 raft_consensus.cc:504] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:14.114624 28643 leader_election.cc:288] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [CANDIDATE]: Term 1 election: Requested vote from peers d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835), c27b291221ae41158d078f84c3e2811e (127.7.164.65:46853)
I20260808 18:30:14.115049 28548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289c180b0a454ed69b9677104627d239" candidate_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0d96fd4d43c4b25b304d804885f6bee"
I20260808 18:30:14.115127 28176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289c180b0a454ed69b9677104627d239" candidate_uuid: "d0d96fd4d43c4b25b304d804885f6bee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c27b291221ae41158d078f84c3e2811e" is_pre_election: true
I20260808 18:30:14.115187 28548 raft_consensus.cc:3037] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:14.115268 28176 raft_consensus.cc:2454] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0d96fd4d43c4b25b304d804885f6bee in term 0.
I20260808 18:30:14.115396 28362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289c180b0a454ed69b9677104627d239" candidate_uuid: "d0d96fd4d43c4b25b304d804885f6bee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" is_pre_election: true
I20260808 18:30:14.115607 28362 raft_consensus.cc:2379] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d0d96fd4d43c4b25b304d804885f6bee in current term 1: Already voted for candidate bbe65ce30d3144c0a865e4f2bfa4162c in this term.
I20260808 18:30:14.116176 28461 leader_election.cc:302] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c27b291221ae41158d078f84c3e2811e, d0d96fd4d43c4b25b304d804885f6bee; no voters: bbe65ce30d3144c0a865e4f2bfa4162c
I20260808 18:30:14.116494 28548 raft_consensus.cc:2454] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbe65ce30d3144c0a865e4f2bfa4162c in term 1.
I20260808 18:30:14.116602 28644 raft_consensus.cc:2741] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260808 18:30:14.116766 28177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289c180b0a454ed69b9677104627d239" candidate_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c27b291221ae41158d078f84c3e2811e"
I20260808 18:30:14.116772 28274 leader_election.cc:302] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbe65ce30d3144c0a865e4f2bfa4162c, d0d96fd4d43c4b25b304d804885f6bee; no voters:
I20260808 18:30:14.116882 28177 raft_consensus.cc:3037] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:14.116961 28643 raft_consensus.cc:2781] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:30:14.117194 28643 raft_consensus.cc:686] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 1 LEADER]: Becoming Leader. State: Replica: bbe65ce30d3144c0a865e4f2bfa4162c, State: Running, Role: LEADER
I20260808 18:30:14.117559 28643 consensus_queue.cc:237] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:14.118193 28177 raft_consensus.cc:2454] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbe65ce30d3144c0a865e4f2bfa4162c in term 1.
I20260808 18:30:14.121239 27964 catalog_manager.cc:5495] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c reported cstate change: term changed from 0 to 1, leader changed from <none> to bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66). New cstate: current_term: 1 leader_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } health_report { overall_health: HEALTHY } } }
W20260808 18:30:14.133157 28612 tablet.cc:2369] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:30:14.149482 28428 tablet.cc:2369] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:30:14.167236 28243 tablet.cc:2369] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:30:14.172513 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.169264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38310 (local address 127.7.164.66:41509)
0808 18:30:14.169397 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:14.169400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:14.169414 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:30:14.169492 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:14.169494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:14.169557 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:14.169647 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:30:14.169876 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.170439 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.171238 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.171447 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.171530 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.171785 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:30:14.172201 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:30:14.172294 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:14.172347 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 18:30:14.172392 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:30:14.178301 28548 raft_consensus.cc:1261] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [term 1 FOLLOWER]: Refusing update from remote peer bbe65ce30d3144c0a865e4f2bfa4162c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:30:14.178300 28177 raft_consensus.cc:1261] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Refusing update from remote peer bbe65ce30d3144c0a865e4f2bfa4162c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:30:14.179040 28643 consensus_queue.cc:1035] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:30:14.179302 28650 consensus_queue.cc:1035] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:30:14.186269 28655 mvcc.cc:205] Tried to move back new op lower bound from 7316331782866694144 to 7316331782629183488. Current Snapshot: MvccSnapshot[applied={T|T < 7316331782866694144}]
I20260808 18:30:14.196738 27959 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:30:14.196938 27959 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:30:14.199584 27959 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Sending DeleteTablet for 3 replicas of tablet 289c180b0a454ed69b9677104627d239
I20260808 18:30:14.200501 28335 tablet_service.cc:1514] Processing DeleteTablet for tablet 289c180b0a454ed69b9677104627d239 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:14 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:38298
I20260808 18:30:14.200685 28149 tablet_service.cc:1514] Processing DeleteTablet for tablet 289c180b0a454ed69b9677104627d239 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:14 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:34774
I20260808 18:30:14.200850 28524 tablet_service.cc:1514] Processing DeleteTablet for tablet 289c180b0a454ed69b9677104627d239 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:14 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:50368
I20260808 18:30:14.201254 28669 tablet_replica.cc:330] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: stopping tablet replica
I20260808 18:30:14.201434 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 28054
I20260808 18:30:14.214593 28672 tablet_replica.cc:330] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee: stopping tablet replica
I20260808 18:30:14.215091 28672 raft_consensus.cc:2229] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:30:14.215499 28672 raft_consensus.cc:2258] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:30:14.215969 28668 tablet_replica.cc:330] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: stopping tablet replica
I20260808 18:30:14.216367 28668 raft_consensus.cc:2229] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:30:14.216873 28668 raft_consensus.cc:2258] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:30:14.217357 28672 ts_tablet_manager.cc:1905] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 18:30:14.218442 27942 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:46853 (error 108)
I20260808 18:30:14.219070 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 28250
I20260808 18:30:14.222724 28672 ts_tablet_manager.cc:1918] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:30:14.222862 28672 log.cc:1196] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/289c180b0a454ed69b9677104627d239
I20260808 18:30:14.223181 28672 ts_tablet_manager.cc:1939] T 289c180b0a454ed69b9677104627d239 P d0d96fd4d43c4b25b304d804885f6bee: Deleting consensus metadata
I20260808 18:30:14.226915 27942 catalog_manager.cc:4841] TS d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835): tablet 289c180b0a454ed69b9677104627d239 (table test-table [id=e61be8594f0f4cf198025ac2670a32ce]) successfully deleted
I20260808 18:30:14.227517 28628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.226682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:46853 (local address 127.0.0.1:34784)
0808 18:30:14.227290 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:14.227401 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:46853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484}
W20260808 18:30:14.227718 27942 catalog_manager.cc:4584] TS c27b291221ae41158d078f84c3e2811e (127.7.164.65:46853): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 289c180b0a454ed69b9677104627d239: Network error: Client connection negotiation failed: client connection to 127.7.164.65:46853: connect: Connection refused (error 111)
I20260808 18:30:14.233819 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 28435
I20260808 18:30:14.238663 28628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.233871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:41509 (local address 127.0.0.1:38324)
0808 18:30:14.234109 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:14.234170 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.66:41509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260808 18:30:14.244131 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 27920
I20260808 18:30:14.254496 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:45267
--webserver_interface=127.7.164.126
--webserver_port=36539
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:45267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:14.381876 28674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:14.382186 28674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:14.382287 28674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:14.386476 28674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:30:14.386572 28674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:14.386611 28674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:14.386678 28674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:30:14.386729 28674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:30:14.392004 28674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:45267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:45267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=36539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:14.393105 28674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:14.394150 28674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:14.408823 28686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:14.408825 28684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:14.408825 28683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:14.409301 28674 server_base.cc:1034] running on GCE node
I20260808 18:30:14.410006 28674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:14.411337 28674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:14.412554 28674 hybrid_clock.cc:648] HybridClock initialized: now 1786213814412532 us; error 28 us; skew 500 ppm
Aug 08 18:30:14 dist-test-slave-jztv krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213814, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:14.415565 28674 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:30:14.416594 28674 webserver.cc:460] Webserver started at http://127.7.164.126:36539/ using document root <none> and password file <none>
I20260808 18:30:14.417225 28674 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:14.417346 28674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:14.421485 28674 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:30:14.423787 28696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:14.424968 28674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:30:14.425136 28674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "8962b17c345d40fd9c9c39e1b5d70f9a"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "271471d1d91d7e241e6f3f2dc79e15d5"
server_key_iv: "beb9974d4c9ea8bbdc619a5255ffad63"
server_key_version: "encryptionkey@0"
I20260808 18:30:14.425765 28674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:14.438195 28674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:14.441129 28674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:14.441349 28674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:14.448927 28674 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:45267
I20260808 18:30:14.449225 28767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:45267 every 8 connection(s)
I20260808 18:30:14.450075 28674 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:30:14.453318 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 28674
I20260808 18:30:14.453763 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:46853
--local_ip_for_outbound_sockets=127.7.164.65
--tserver_master_addrs=127.7.164.126:45267
--webserver_port=43107
--webserver_interface=127.7.164.65
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:30:14.454597 28769 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:30:14.456230 28769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Bootstrap starting.
I20260808 18:30:14.481412 28769 log.cc:824] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:14.492583 28769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:30:14.493052 28769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Bootstrap complete.
I20260808 18:30:14.498663 28769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } }
I20260808 18:30:14.499243 28769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8962b17c345d40fd9c9c39e1b5d70f9a, State: Initialized, Role: FOLLOWER
I20260808 18:30:14.499890 28769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } }
I20260808 18:30:14.500057 28769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:30:14.500137 28769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:30:14.500233 28769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:30:14.502874 28769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } }
I20260808 18:30:14.503270 28769 leader_election.cc:302] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8962b17c345d40fd9c9c39e1b5d70f9a; no voters:
I20260808 18:30:14.503618 28769 leader_election.cc:288] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 18:30:14.503757 28773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 2 FOLLOWER]: Leader election won for term 2
I20260808 18:30:14.504070 28773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 2 LEADER]: Becoming Leader. State: Replica: 8962b17c345d40fd9c9c39e1b5d70f9a, State: Running, Role: LEADER
I20260808 18:30:14.504400 28773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } }
I20260808 18:30:14.504895 28769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:30:14.505604 28775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8962b17c345d40fd9c9c39e1b5d70f9a. Latest consensus state: current_term: 2 leader_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } } }
I20260808 18:30:14.505797 28775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:14.505867 28774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } } }
I20260808 18:30:14.505949 28774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:14.506309 28782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:30:14.511621 28782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:30:14.512363 28782 catalog_manager.cc:1259] Loaded cluster ID: 9b8a07200a6d40ad8672f352fbcab626
I20260808 18:30:14.512450 28782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:30:14.513604 28782 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 18:30:14.516445 28782 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 18:30:14.521610 28782 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:30:14.522401 28782 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Loaded TSK: 0
I20260808 18:30:14.523422 28782 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:30:14.599465 28771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:14.599784 28771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:14.599864 28771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:14.604316 28771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:14.604419 28771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:14.604547 28771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:30:14.610589 28771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:46853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=43107
--tserver_master_addrs=127.7.164.126:45267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:14.611696 28771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:14.612728 28771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:14.622862 28815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:14.622859 28817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:14.622859 28813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:14.623350 28771 server_base.cc:1034] running on GCE node
I20260808 18:30:14.623768 28771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:14.624472 28771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:14.625669 28771 hybrid_clock.cc:648] HybridClock initialized: now 1786213814625636 us; error 48 us; skew 500 ppm
Aug 08 18:30:14 dist-test-slave-jztv krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213814, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:14.628713 28771 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:30:14.629766 28771 webserver.cc:460] Webserver started at http://127.7.164.65:43107/ using document root <none> and password file <none>
I20260808 18:30:14.630442 28771 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:14.630566 28771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:14.634910 28771 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:30:14.637954 28828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:14.639391 28771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:30:14.639526 28771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "c27b291221ae41158d078f84c3e2811e"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "732405baeac4196a708b3d57143b397c"
server_key_iv: "c8fd6a91464c6ebb7e2aec4188470df3"
server_key_version: "encryptionkey@0"
I20260808 18:30:14.640136 28771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:14.668447 28771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:14.671579 28771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:14.671799 28771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:14.672442 28771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:14.673611 28837 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 18:30:14.678146 28771 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 18:30:14.678292 28771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260808 18:30:14.678400 28771 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 18:30:14.680115 28771 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 18:30:14.680164 28771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 18:30:14.680538 28837 tablet_bootstrap.cc:492] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Bootstrap starting.
I20260808 18:30:14.691011 28771 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:46853
I20260808 18:30:14.691107 28981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:46853 every 8 connection(s)
I20260808 18:30:14.692021 28771 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:30:14.693172 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 28771
I20260808 18:30:14.693606 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:41509
--local_ip_for_outbound_sockets=127.7.164.66
--tserver_master_addrs=127.7.164.126:45267
--webserver_port=40615
--webserver_interface=127.7.164.66
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:30:14.701817 28837 log.cc:824] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:30:14 dist-test-slave-jztv krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213814, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:14.710752 28837 tablet_bootstrap.cc:492] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:30:14.711231 28837 tablet_bootstrap.cc:492] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Bootstrap complete.
I20260808 18:30:14.712764 28837 ts_tablet_manager.cc:1397] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Time spent bootstrapping tablet: real 0.032s user 0.025s sys 0.004s
I20260808 18:30:14.713937 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.695424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:57181)
0808 18:30:14.696506 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:14.696571 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:14.697527 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:14.699055 (+ 1528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:14.699068 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:14.699678 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:14.700436 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.700462 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.702435 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.702438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.703632 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.703644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.704834 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.704840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.705031 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.705796 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:14.705817 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:14.707878 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:14.710423 (+ 2545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:14.710429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:14.710442 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:14.710777 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:14.711256 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:14.711260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:14.711262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:14.711444 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:14.712194 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:14.712200 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:14.712481 (+ 281us) client_negotiation.cc:769] Sending connection context
0808 18:30:14.712685 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 18:30:14.713011 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"thread_start_us":165,"threads_started":1}
I20260808 18:30:14.714375 28989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.695865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:57181 (local address 127.7.164.126:45267)
0808 18:30:14.696739 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:14.696767 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:14.696824 (+ 57us) server_negotiation.cc:407] Connection header received
0808 18:30:14.698052 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:14.698075 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:14.698563 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:14.698952 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:14.700676 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.702136 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.703772 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.704613 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.704823 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.708109 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:14.708116 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:14.708132 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:14.708166 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:14.710282 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:14.710945 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:14.710954 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:14.710961 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:14.711129 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:14.711560 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:14.711567 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:14.711571 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:14.712058 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:14.712190 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:14.712910 (+ 720us) server_negotiation.cc:299] Negotiation successful
0808 18:30:14.713222 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":135,"threads_started":1}
I20260808 18:30:14.715607 28837 raft_consensus.cc:348] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:14.716122 28837 raft_consensus.cc:729] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c27b291221ae41158d078f84c3e2811e, State: Initialized, Role: FOLLOWER
I20260808 18:30:14.716620 28837 consensus_queue.cc:260] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:14.717286 28983 heartbeater.cc:347] Connected to a master server at 127.7.164.126:45267
I20260808 18:30:14.717522 28983 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:14.717701 28837 ts_tablet_manager.cc:1428] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260808 18:30:14.718060 28983 heartbeater.cc:511] Master 127.7.164.126:45267 requested a full tablet report, sending...
I20260808 18:30:14.720611 28719 ts_manager.cc:194] Registered new tserver with Master: c27b291221ae41158d078f84c3e2811e (127.7.164.65:46853)
I20260808 18:30:14.724053 28719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:57181
I20260808 18:30:14.726672 28983 heartbeater.cc:503] Master 127.7.164.126:45267 was elected leader, sending a full tablet report...
Aug 08 18:30:14 dist-test-slave-jztv krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213814, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for kudu/127.7.164.65@KRBTEST.COM
I20260808 18:30:14.734598 28994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.723391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:46853 (local address 127.0.0.1:34796)
0808 18:30:14.724139 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:14.724185 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:14.724372 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:14.724929 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:14.724937 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:14.725356 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:14.725497 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.725517 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.726926 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.726929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.727673 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.727682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.728437 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.728441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.728550 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.729126 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:14.729142 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:14.730904 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:14.732789 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:14.732798 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:14.732803 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:14.733035 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:14.733347 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:14.733351 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:14.733353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:14.733451 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:14.733923 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:14.733929 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:14.734055 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 18:30:14.734197 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 18:30:14.734415 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":114,"threads_started":1}
I20260808 18:30:14.734611 28995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.723475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.7.164.65:46853)
0808 18:30:14.723875 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:14.723887 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:14.724268 (+ 381us) server_negotiation.cc:407] Connection header received
0808 18:30:14.724547 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:14.724552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:14.724752 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:14.724889 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:14.725650 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.726777 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.727800 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.728165 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.728269 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.731043 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:14.731053 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:14.731071 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:14.731092 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:14.732657 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:14.733153 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:14.733158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:14.733160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:14.733249 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:14.733545 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:14.733550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:14.733551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:14.733784 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:14.733944 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:14.734305 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 18:30:14.734468 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":315,"thread_start_us":128,"threads_started":1}
I20260808 18:30:14.735673 28893 tablet_service.cc:1514] Processing DeleteTablet for tablet 289c180b0a454ed69b9677104627d239 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:14 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:34796
I20260808 18:30:14.736723 28998 tablet_replica.cc:330] stopping tablet replica
I20260808 18:30:14.737025 28998 raft_consensus.cc:2229] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:30:14.737284 28998 raft_consensus.cc:2258] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:30:14.738320 28998 ts_tablet_manager.cc:1905] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:30:14.741741 28998 ts_tablet_manager.cc:1918] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:30:14.741858 28998 log.cc:1196] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal/wals/289c180b0a454ed69b9677104627d239
I20260808 18:30:14.742159 28998 ts_tablet_manager.cc:1939] T 289c180b0a454ed69b9677104627d239 P c27b291221ae41158d078f84c3e2811e: Deleting consensus metadata
I20260808 18:30:14.743834 28699 catalog_manager.cc:4841] TS c27b291221ae41158d078f84c3e2811e (127.7.164.65:46853): tablet 289c180b0a454ed69b9677104627d239 (table test-table [id=e61be8594f0f4cf198025ac2670a32ce]) successfully deleted
W20260808 18:30:14.819466 28985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:14.819732 28985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:14.819775 28985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:14.823930 28985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:14.823997 28985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:14.824057 28985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:30:14.829439 28985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:41509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=40615
--tserver_master_addrs=127.7.164.126:45267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:14.830610 28985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:14.831593 28985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:14.841382 29006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:14.841375 29008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:14.841375 29005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:14.841657 28985 server_base.cc:1034] running on GCE node
I20260808 18:30:14.842116 28985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:14.842787 28985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:14.843942 28985 hybrid_clock.cc:648] HybridClock initialized: now 1786213814843932 us; error 33 us; skew 500 ppm
Aug 08 18:30:14 dist-test-slave-jztv krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213814, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:14.846650 28985 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:30:14.847566 28985 webserver.cc:460] Webserver started at http://127.7.164.66:40615/ using document root <none> and password file <none>
I20260808 18:30:14.848137 28985 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:14.848248 28985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:14.852442 28985 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:30:14.855118 29016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:14.856312 28985 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:30:14.856459 28985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "bbe65ce30d3144c0a865e4f2bfa4162c"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "328fa7c30a4758b6fce308172c983704"
server_key_iv: "fa58761314c44a094881a986e5a78304"
server_key_version: "encryptionkey@0"
I20260808 18:30:14.857049 28985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:14.867919 28985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:14.871250 28985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:14.871461 28985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:14.872108 28985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:14.873281 29027 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 18:30:14.877561 28985 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 18:30:14.877638 28985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260808 18:30:14.877734 28985 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 18:30:14.879633 28985 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 18:30:14.879681 28985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 18:30:14.879990 29027 tablet_bootstrap.cc:492] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Bootstrap starting.
I20260808 18:30:14.891024 28985 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:41509
I20260808 18:30:14.891099 29189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:41509 every 8 connection(s)
I20260808 18:30:14.892052 28985 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:30:14.893034 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 28985
I20260808 18:30:14.893563 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:39835
--local_ip_for_outbound_sockets=127.7.164.67
--tserver_master_addrs=127.7.164.126:45267
--webserver_port=35047
--webserver_interface=127.7.164.67
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:30:14.899026 29027 log.cc:824] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:30:14 dist-test-slave-jztv krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213814, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:14.907969 29027 tablet_bootstrap.cc:492] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:30:14.908499 29027 tablet_bootstrap.cc:492] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Bootstrap complete.
I20260808 18:30:14.909471 28989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.894094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:58171 (local address 127.7.164.126:45267)
0808 18:30:14.894264 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:14.894269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:14.894976 (+ 707us) server_negotiation.cc:407] Connection header received
0808 18:30:14.896213 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:14.896217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:14.896282 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:14.896391 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:14.897921 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.898708 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.900653 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.900992 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.901107 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.904731 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:14.904736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:14.904740 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:14.904775 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:14.906756 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:14.907494 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:14.907498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:14.907499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:14.907607 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:14.908072 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:14.908076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:14.908078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:14.908356 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:14.908476 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:14.909055 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 18:30:14.909220 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:30:14.910040 29027 ts_tablet_manager.cc:1397] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Time spent bootstrapping tablet: real 0.030s user 0.030s sys 0.000s
I20260808 18:30:14.910310 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.894280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:58171)
0808 18:30:14.894805 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:14.894869 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:14.895931 (+ 1062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:14.896596 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:14.896608 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:14.897175 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:14.897743 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.897758 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.898884 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.898890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.900499 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.900516 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.901157 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.901163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.901425 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.902186 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:14.902242 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:14.904526 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:14.906894 (+ 2368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:14.906902 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:14.906919 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:14.907328 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:14.907743 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:14.907749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:14.907754 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:14.907948 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:14.908480 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:14.908486 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:14.908782 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 18:30:14.908980 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 18:30:14.909318 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":156,"threads_started":1}
I20260808 18:30:14.911562 29190 heartbeater.cc:347] Connected to a master server at 127.7.164.126:45267
I20260808 18:30:14.911813 29190 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:14.912361 29190 heartbeater.cc:511] Master 127.7.164.126:45267 requested a full tablet report, sending...
I20260808 18:30:14.913060 29027 raft_consensus.cc:348] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:14.913590 29027 raft_consensus.cc:729] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe65ce30d3144c0a865e4f2bfa4162c, State: Initialized, Role: FOLLOWER
I20260808 18:30:14.914093 29027 consensus_queue.cc:260] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:14.914273 28719 ts_manager.cc:194] Registered new tserver with Master: bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509)
I20260808 18:30:14.915221 29027 ts_tablet_manager.cc:1428] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260808 18:30:14.915577 28719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:58171
I20260808 18:30:14.917639 29190 heartbeater.cc:503] Master 127.7.164.126:45267 was elected leader, sending a full tablet report...
Aug 08 18:30:14 dist-test-slave-jztv krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213814, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for kudu/127.7.164.66@KRBTEST.COM
I20260808 18:30:14.925419 28994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.915257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:41509 (local address 127.0.0.1:38330)
0808 18:30:14.915854 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:14.915870 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:14.915957 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:14.916461 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:14.916465 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:14.916661 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:14.916765 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.916772 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.918380 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.918385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.919348 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:14.919367 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.919916 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.919922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:14.920024 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.920499 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:14.920513 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:14.921852 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:14.923765 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:14.923770 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:14.923772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:14.923981 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:14.924288 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:14.924291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:14.924293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:14.924391 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:14.924859 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:14.924863 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:14.924970 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 18:30:14.925081 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 18:30:14.925238 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515}
I20260808 18:30:14.925455 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:14.915541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38330 (local address 127.7.164.66:41509)
0808 18:30:14.915999 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:14.916012 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:14.916034 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:30:14.916102 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:14.916107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:14.916305 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:14.916449 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:14.917043 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.918233 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.919475 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:14.919800 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:14.919884 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:14.921994 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:14.922004 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:14.922024 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:14.922046 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:14.923634 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:14.924098 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:14.924111 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:14.924112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:14.924189 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:14.924492 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:14.924497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:14.924498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:14.924740 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:14.924879 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:14.925166 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 18:30:14.925310 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":104,"threads_started":1}
I20260808 18:30:14.926455 29086 tablet_service.cc:1514] Processing DeleteTablet for tablet 289c180b0a454ed69b9677104627d239 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:14 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:38330
I20260808 18:30:14.927429 29201 tablet_replica.cc:330] stopping tablet replica
I20260808 18:30:14.927721 29201 raft_consensus.cc:2229] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:30:14.927979 29201 raft_consensus.cc:2258] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:30:14.928977 29201 ts_tablet_manager.cc:1905] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:30:14.932788 29201 ts_tablet_manager.cc:1918] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:30:14.932919 29201 log.cc:1196] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/289c180b0a454ed69b9677104627d239
I20260808 18:30:14.933203 29201 ts_tablet_manager.cc:1939] T 289c180b0a454ed69b9677104627d239 P bbe65ce30d3144c0a865e4f2bfa4162c: Deleting consensus metadata
I20260808 18:30:14.934862 28703 catalog_manager.cc:4841] TS bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509): tablet 289c180b0a454ed69b9677104627d239 (table test-table [id=e61be8594f0f4cf198025ac2670a32ce]) successfully deleted
W20260808 18:30:15.021510 29196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:15.021755 29196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:15.021798 29196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:15.025962 29196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:15.026018 29196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:15.026077 29196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:30:15.031471 29196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:39835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=35047
--tserver_master_addrs=127.7.164.126:45267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:15.032483 29196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:15.033438 29196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:15.043123 29208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:15.043212 29211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:15.043283 29213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:15.043526 29196 server_base.cc:1034] running on GCE node
I20260808 18:30:15.043941 29196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:15.044620 29196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:15.045804 29196 hybrid_clock.cc:648] HybridClock initialized: now 1786213815045783 us; error 39 us; skew 500 ppm
Aug 08 18:30:15 dist-test-slave-jztv krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213815, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:15.048496 29196 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:30:15.049280 29196 webserver.cc:460] Webserver started at http://127.7.164.67:35047/ using document root <none> and password file <none>
I20260808 18:30:15.049773 29196 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:15.049841 29196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:15.053460 29196 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:30:15.055480 29222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:15.056661 29196 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:30:15.056783 29196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "d0d96fd4d43c4b25b304d804885f6bee"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "60052cddb8e7954e48c110504325204b"
server_key_iv: "9c37dfe00b6b257db78ddfcdc2d36d07"
server_key_version: "encryptionkey@0"
I20260808 18:30:15.057307 29196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:15.080920 29196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:15.083938 29196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:15.084143 29196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:15.084817 29196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:15.085662 29196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:15.085711 29196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:15.085777 29196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:15.085811 29196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:15.094954 29196 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:39835
I20260808 18:30:15.094981 29396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:39835 every 8 connection(s)
I20260808 18:30:15.095992 29196 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:30:15.103403 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 29196
Aug 08 18:30:15 dist-test-slave-jztv krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213815, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:15.111311 28989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.097951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:45301 (local address 127.7.164.126:45267)
0808 18:30:15.098120 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:15.098124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:15.099284 (+ 1160us) server_negotiation.cc:407] Connection header received
0808 18:30:15.100390 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:15.100394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:15.100455 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:15.100574 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:15.101790 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.102425 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.103873 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.104207 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.104342 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:15.107094 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:15.107099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:15.107101 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:15.107131 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:15.108772 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:15.109335 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:15.109339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:15.109341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:15.109438 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:15.109851 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:15.109855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:15.109856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:15.110083 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:15.110164 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:15.111036 (+ 872us) server_negotiation.cc:299] Negotiation successful
0808 18:30:15.111183 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 18:30:15.112478 29401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.098385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:45301)
0808 18:30:15.099115 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:15.099158 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:15.100150 (+ 992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:15.100768 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:15.100775 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:15.101147 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:15.101633 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:15.101648 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.102559 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.102562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:15.103721 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:15.103733 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.104401 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.104404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:15.104551 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:15.105229 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:15.105247 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:15.106916 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:15.108867 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:15.108874 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:15.108894 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:15.109225 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:15.109593 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:15.109596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:15.109597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:15.109744 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:15.110321 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:15.110329 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:15.110713 (+ 384us) client_negotiation.cc:769] Sending connection context
0808 18:30:15.110964 (+ 251us) client_negotiation.cc:240] Negotiation successful
0808 18:30:15.111374 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":122,"threads_started":1}
I20260808 18:30:15.113646 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.104157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39754 (local address 127.7.164.126:45267)
0808 18:30:15.104983 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:15.104990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:15.105005 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:30:15.105044 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:15.105048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:15.105094 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:15.105192 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:15.105966 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.106655 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.108003 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.108370 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.108484 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:15.110003 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:15.110006 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:15.110007 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:15.110031 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:15.111864 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:15.112279 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:15.112283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:15.112284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:15.112379 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:15.112651 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:15.112655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:15.112657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:15.112901 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:15.113118 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:15.113334 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 18:30:15.113487 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":102,"threads_started":1}
I20260808 18:30:15.113906 29397 heartbeater.cc:347] Connected to a master server at 127.7.164.126:45267
I20260808 18:30:15.114177 29397 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:15.114823 29397 heartbeater.cc:511] Master 127.7.164.126:45267 requested a full tablet report, sending...
I20260808 18:30:15.115906 28719 ts_manager.cc:194] Registered new tserver with Master: d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835)
I20260808 18:30:15.116458 28719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:45301
I20260808 18:30:15.127466 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:30:15.131793 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.128136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39766 (local address 127.7.164.126:45267)
0808 18:30:15.128262 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:15.128265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:15.128470 (+ 205us) server_negotiation.cc:407] Connection header received
0808 18:30:15.128606 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:15.128608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:15.128708 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:15.128796 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:30:15.129041 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.129561 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.130438 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.130657 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.130734 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:15.131016 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:30:15.131438 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:30:15.131532 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:15.131600 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 18:30:15.131656 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:30:15.132432 28719 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:30:15.135695 28719 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:30:15.143657 28893 tablet_service.cc:1467] Processing CreateTablet for tablet b8ecf88f4162411997521cca77a8df89 (DEFAULT_TABLE table=test-table [id=a3bd304fcb7e41368aefd3bf5f93188a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:15.143657 29086 tablet_service.cc:1467] Processing CreateTablet for tablet b8ecf88f4162411997521cca77a8df89 (DEFAULT_TABLE table=test-table [id=a3bd304fcb7e41368aefd3bf5f93188a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:15.144326 28893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8ecf88f4162411997521cca77a8df89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:15.144325 29086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8ecf88f4162411997521cca77a8df89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:15.147349 28837 tablet_bootstrap.cc:492] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e: Bootstrap starting.
I20260808 18:30:15.148288 28837 tablet_bootstrap.cc:654] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:15.148545 29027 tablet_bootstrap.cc:492] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c: Bootstrap starting.
I20260808 18:30:15.148721 28994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.142847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:39835 (local address 127.0.0.1:50370)
0808 18:30:15.143053 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:15.143071 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:15.143163 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:15.144293 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:15.144297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:15.144347 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:15.144541 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:15.144557 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.146077 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.146081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:15.146955 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:15.146966 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.148203 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.148207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:15.148315 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:15.148396 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 18:30:15.148481 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 18:30:15.148575 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260808 18:30:15.149169 29413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.143014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50370 (local address 127.7.164.67:39835)
0808 18:30:15.143461 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:15.143473 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:15.143503 (+ 30us) server_negotiation.cc:407] Connection header received
0808 18:30:15.143572 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:15.143577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:15.143773 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:15.143995 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:15.144721 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.145887 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.147088 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.148083 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.148211 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:15.148388 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:15.148525 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 18:30:15.148691 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":141,"threads_started":1}
I20260808 18:30:15.149525 29027 tablet_bootstrap.cc:654] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:15.149827 28837 tablet_bootstrap.cc:492] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e: No bootstrap required, opened a new log
I20260808 18:30:15.149887 28837 ts_tablet_manager.cc:1397] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:30:15.150321 28837 raft_consensus.cc:348] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:15.150460 28837 raft_consensus.cc:374] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:15.150504 28837 raft_consensus.cc:729] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c27b291221ae41158d078f84c3e2811e, State: Initialized, Role: FOLLOWER
I20260808 18:30:15.150656 28837 consensus_queue.cc:260] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:15.150782 28992 raft_consensus.cc:482] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:30:15.150971 28992 raft_consensus.cc:504] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:15.151032 28837 ts_tablet_manager.cc:1428] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:30:15.152179 28992 leader_election.cc:288] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835), bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509)
I20260808 18:30:15.152454 29027 tablet_bootstrap.cc:492] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c: No bootstrap required, opened a new log
I20260808 18:30:15.152539 29027 ts_tablet_manager.cc:1397] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 18:30:15.152635 29299 tablet_service.cc:1467] Processing CreateTablet for tablet b8ecf88f4162411997521cca77a8df89 (DEFAULT_TABLE table=test-table [id=a3bd304fcb7e41368aefd3bf5f93188a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:15.153087 29027 raft_consensus.cc:348] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:15.153205 29027 raft_consensus.cc:374] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:15.153249 29027 raft_consensus.cc:729] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe65ce30d3144c0a865e4f2bfa4162c, State: Initialized, Role: FOLLOWER
I20260808 18:30:15.153371 29027 consensus_queue.cc:260] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:15.153671 29027 ts_tablet_manager.cc:1428] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:30:15.153831 29299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8ecf88f4162411997521cca77a8df89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:15.157420 29413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.153280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:40027 (local address 127.7.164.67:39835)
0808 18:30:15.153394 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:15.153398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:15.153423 (+ 25us) server_negotiation.cc:407] Connection header received
0808 18:30:15.153467 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:15.153470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:15.153514 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:15.153918 (+ 404us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:15.154023 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.154766 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.156251 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.156829 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.156960 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:15.157011 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:15.157227 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 18:30:15.157310 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:30:15.157442 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.152519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:39835 (local address 127.7.164.65:40027)
0808 18:30:15.152644 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:15.152663 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:15.152758 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:15.153643 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:15.153646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:15.153667 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:15.153796 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:15.153807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.154886 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.154890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:15.156098 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:15.156109 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.156953 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.156956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:15.157055 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:15.157096 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 18:30:15.157210 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 18:30:15.157291 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 18:30:15.159148 28831 leader_election.cc:341] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835): Not found: Tablet not found: b8ecf88f4162411997521cca77a8df89
I20260808 18:30:15.160590 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.152795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:41509 (local address 127.7.164.65:44307)
0808 18:30:15.154474 (+ 1679us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:15.154487 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:15.154611 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:15.154969 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:15.154971 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:15.154987 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:15.157177 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:15.157185 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.158176 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.158180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:15.159356 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:15.159368 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.160130 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.160134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:15.160241 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:15.160266 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:30:15.160364 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 18:30:15.160446 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1615,"mutex_wait_us":2036,"thread_start_us":115,"threads_started":1}
I20260808 18:30:15.160620 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.152993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:44307 (local address 127.7.164.66:41509)
0808 18:30:15.153168 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:15.153172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:15.154553 (+ 1381us) server_negotiation.cc:407] Connection header received
0808 18:30:15.154707 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:15.154710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:15.154757 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:15.154820 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:15.157318 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.158029 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.159524 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.159986 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.160093 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:15.160267 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:15.160392 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 18:30:15.160496 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 18:30:15.161099 29114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8ecf88f4162411997521cca77a8df89" candidate_uuid: "c27b291221ae41158d078f84c3e2811e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" is_pre_election: true
I20260808 18:30:15.161435 29114 raft_consensus.cc:2454] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c27b291221ae41158d078f84c3e2811e in term 0.
I20260808 18:30:15.161899 28832 leader_election.cc:302] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bbe65ce30d3144c0a865e4f2bfa4162c, c27b291221ae41158d078f84c3e2811e; no voters: d0d96fd4d43c4b25b304d804885f6bee
I20260808 18:30:15.162182 28992 raft_consensus.cc:2781] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:30:15.162307 28992 raft_consensus.cc:482] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:30:15.162359 28992 raft_consensus.cc:3037] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:15.163378 28992 raft_consensus.cc:504] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:15.163803 28992 leader_election.cc:288] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 1 election: Requested vote from peers d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835), bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509)
I20260808 18:30:15.163789 29423 tablet_bootstrap.cc:492] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee: Bootstrap starting.
I20260808 18:30:15.164220 29114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8ecf88f4162411997521cca77a8df89" candidate_uuid: "c27b291221ae41158d078f84c3e2811e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c"
I20260808 18:30:15.164357 29114 raft_consensus.cc:3037] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:15.164384 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8ecf88f4162411997521cca77a8df89" candidate_uuid: "c27b291221ae41158d078f84c3e2811e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0d96fd4d43c4b25b304d804885f6bee"
I20260808 18:30:15.165786 29114 raft_consensus.cc:2454] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c27b291221ae41158d078f84c3e2811e in term 1.
W20260808 18:30:15.165817 28831 leader_election.cc:341] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835): Illegal state: must be running to vote when last-logged opid is not known
I20260808 18:30:15.166113 28832 leader_election.cc:302] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bbe65ce30d3144c0a865e4f2bfa4162c, c27b291221ae41158d078f84c3e2811e; no voters: d0d96fd4d43c4b25b304d804885f6bee
I20260808 18:30:15.166365 28992 raft_consensus.cc:2781] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:30:15.166474 28992 raft_consensus.cc:686] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 LEADER]: Becoming Leader. State: Replica: c27b291221ae41158d078f84c3e2811e, State: Running, Role: LEADER
I20260808 18:30:15.166724 28992 consensus_queue.cc:237] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:15.166874 29423 tablet_bootstrap.cc:654] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:15.167814 29423 log.cc:824] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:15.170560 28717 catalog_manager.cc:5495] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e reported cstate change: term changed from 0 to 1, leader changed from <none> to c27b291221ae41158d078f84c3e2811e (127.7.164.65). New cstate: current_term: 1 leader_uuid: "c27b291221ae41158d078f84c3e2811e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } health_report { overall_health: UNKNOWN } } }
I20260808 18:30:15.173491 29423 tablet_bootstrap.cc:492] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee: No bootstrap required, opened a new log
I20260808 18:30:15.173759 29423 ts_tablet_manager.cc:1397] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260808 18:30:15.177033 29423 raft_consensus.cc:348] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:15.177289 29423 raft_consensus.cc:374] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:15.177333 29423 raft_consensus.cc:729] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0d96fd4d43c4b25b304d804885f6bee, State: Initialized, Role: FOLLOWER
I20260808 18:30:15.177767 29423 consensus_queue.cc:260] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:15.179740 29397 heartbeater.cc:503] Master 127.7.164.126:45267 was elected leader, sending a full tablet report...
I20260808 18:30:15.180156 29423 ts_tablet_manager.cc:1428] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260808 18:30:15.186081 28995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.182070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34806 (local address 127.7.164.65:46853)
0808 18:30:15.182195 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:15.182202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:15.182262 (+ 60us) server_negotiation.cc:407] Connection header received
0808 18:30:15.182352 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:15.182354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:15.182412 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:15.182497 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:30:15.182833 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.183429 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.184601 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:15.184842 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:15.184928 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:15.185256 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:30:15.185748 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:30:15.185840 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:15.185904 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 18:30:15.185958 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 18:30:15.189790 29338 raft_consensus.cc:3037] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:15.189965 29114 raft_consensus.cc:1261] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c [term 1 FOLLOWER]: Refusing update from remote peer c27b291221ae41158d078f84c3e2811e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:30:15.190719 28992 consensus_queue.cc:1035] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:30:15.191349 29338 raft_consensus.cc:1261] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee [term 1 FOLLOWER]: Refusing update from remote peer c27b291221ae41158d078f84c3e2811e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:30:15.191939 28992 consensus_queue.cc:1035] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 18:30:15.193243 28984 tablet.cc:2369] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:30:15.196589 29431 mvcc.cc:205] Tried to move back new op lower bound from 7316331787011870720 to 7316331786926030848. Current Snapshot: MvccSnapshot[applied={T|T < 7316331787011870720}]
I20260808 18:30:15.197702 29433 mvcc.cc:205] Tried to move back new op lower bound from 7316331787011870720 to 7316331786926030848. Current Snapshot: MvccSnapshot[applied={T|T < 7316331787011870720}]
I20260808 18:30:15.205435 29435 mvcc.cc:205] Tried to move back new op lower bound from 7316331787011870720 to 7316331786926030848. Current Snapshot: MvccSnapshot[applied={T|T < 7316331787011870720}]
I20260808 18:30:15.207566 28719 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:30:15.207708 28719 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:30:15.210080 28719 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Sending DeleteTablet for 3 replicas of tablet b8ecf88f4162411997521cca77a8df89
I20260808 18:30:15.210840 28893 tablet_service.cc:1514] Processing DeleteTablet for tablet b8ecf88f4162411997521cca77a8df89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:15 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:34796
I20260808 18:30:15.210887 29086 tablet_service.cc:1514] Processing DeleteTablet for tablet b8ecf88f4162411997521cca77a8df89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:15 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:38330
I20260808 18:30:15.210963 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 28771
I20260808 18:30:15.211051 28998 tablet_replica.cc:330] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e: stopping tablet replica
I20260808 18:30:15.211097 29201 tablet_replica.cc:330] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c: stopping tablet replica
I20260808 18:30:15.211164 28998 raft_consensus.cc:2229] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:30:15.211233 29201 raft_consensus.cc:2229] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:30:15.211378 29201 raft_consensus.cc:2258] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:30:15.220252 29299 tablet_service.cc:1514] Processing DeleteTablet for tablet b8ecf88f4162411997521cca77a8df89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:15 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:50370
I20260808 18:30:15.220628 29201 ts_tablet_manager.cc:1905] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:30:15.220811 29444 tablet_replica.cc:330] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee: stopping tablet replica
I20260808 18:30:15.221120 29444 raft_consensus.cc:2229] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:30:15.221444 29444 raft_consensus.cc:2258] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:30:15.222744 29444 ts_tablet_manager.cc:1905] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:30:15.223670 29201 ts_tablet_manager.cc:1918] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:30:15.223760 29201 log.cc:1196] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/b8ecf88f4162411997521cca77a8df89
I20260808 18:30:15.224037 29201 ts_tablet_manager.cc:1939] T b8ecf88f4162411997521cca77a8df89 P bbe65ce30d3144c0a865e4f2bfa4162c: Deleting consensus metadata
I20260808 18:30:15.224895 28703 catalog_manager.cc:4841] TS bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509): tablet b8ecf88f4162411997521cca77a8df89 (table test-table [id=a3bd304fcb7e41368aefd3bf5f93188a]) successfully deleted
I20260808 18:30:15.225107 29444 ts_tablet_manager.cc:1918] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:30:15.225232 29444 log.cc:1196] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/b8ecf88f4162411997521cca77a8df89
I20260808 18:30:15.225559 29444 ts_tablet_manager.cc:1939] T b8ecf88f4162411997521cca77a8df89 P d0d96fd4d43c4b25b304d804885f6bee: Deleting consensus metadata
I20260808 18:30:15.226698 28699 catalog_manager.cc:4841] TS d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835): tablet b8ecf88f4162411997521cca77a8df89 (table test-table [id=a3bd304fcb7e41368aefd3bf5f93188a]) successfully deleted
I20260808 18:30:15.230319 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 28985
W20260808 18:30:15.230376 28699 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:46853 (error 108)
I20260808 18:30:15.238948 28994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.238500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:46853 (local address 127.0.0.1:34808)
0808 18:30:15.238691 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:15.238806 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:46853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260808 18:30:15.239168 28699 catalog_manager.cc:4584] TS c27b291221ae41158d078f84c3e2811e (127.7.164.65:46853): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b8ecf88f4162411997521cca77a8df89: Network error: Client connection negotiation failed: client connection to 127.7.164.65:46853: connect: Connection refused (error 111)
I20260808 18:30:15.244837 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 29196
I20260808 18:30:15.254504 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 28674
I20260808 18:30:15.265444 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:45267
--webserver_interface=127.7.164.126
--webserver_port=36539
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:45267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:15.380367 29449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:15.380646 29449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:15.380687 29449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:15.384727 29449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:30:15.384789 29449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:15.384809 29449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:15.384825 29449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:30:15.384881 29449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:30:15.390246 29449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:45267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:45267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=36539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:15.391295 29449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:15.392331 29449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:15.401050 29459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:15.401166 29458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:15.401050 29462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:15.401273 29449 server_base.cc:1034] running on GCE node
I20260808 18:30:15.401980 29449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:15.403199 29449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:15.404415 29449 hybrid_clock.cc:648] HybridClock initialized: now 1786213815404371 us; error 54 us; skew 500 ppm
Aug 08 18:30:15 dist-test-slave-jztv krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213815, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:15.407269 29449 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:30:15.408123 29449 webserver.cc:460] Webserver started at http://127.7.164.126:36539/ using document root <none> and password file <none>
I20260808 18:30:15.408718 29449 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:15.408785 29449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:15.412550 29449 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 18:30:15.414619 29472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:15.415799 29449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:30:15.415956 29449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "8962b17c345d40fd9c9c39e1b5d70f9a"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "271471d1d91d7e241e6f3f2dc79e15d5"
server_key_iv: "beb9974d4c9ea8bbdc619a5255ffad63"
server_key_version: "encryptionkey@0"
I20260808 18:30:15.416486 29449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:15.429167 29449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:15.431914 29449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:15.432119 29449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:15.439160 29449 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:45267
I20260808 18:30:15.439483 29548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:45267 every 8 connection(s)
I20260808 18:30:15.440274 29449 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:30:15.442516 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 29449
I20260808 18:30:15.443022 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:46853
--local_ip_for_outbound_sockets=127.7.164.65
--tserver_master_addrs=127.7.164.126:45267
--webserver_port=43107
--webserver_interface=127.7.164.65
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:30:15.445335 29549 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:30:15.447124 29549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Bootstrap starting.
I20260808 18:30:15.465709 29549 log.cc:824] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:15.477245 29549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:30:15.477671 29549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a: Bootstrap complete.
I20260808 18:30:15.481900 29549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } }
I20260808 18:30:15.482399 29549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8962b17c345d40fd9c9c39e1b5d70f9a, State: Initialized, Role: FOLLOWER
I20260808 18:30:15.482895 29549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } }
I20260808 18:30:15.483054 29549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:30:15.483150 29549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:30:15.483271 29549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 2 FOLLOWER]: Advancing to term 3
I20260808 18:30:15.485292 29549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } }
I20260808 18:30:15.485663 29549 leader_election.cc:302] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8962b17c345d40fd9c9c39e1b5d70f9a; no voters:
I20260808 18:30:15.486085 29549 leader_election.cc:288] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 18:30:15.486174 29554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 3 FOLLOWER]: Leader election won for term 3
I20260808 18:30:15.486420 29554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [term 3 LEADER]: Becoming Leader. State: Replica: 8962b17c345d40fd9c9c39e1b5d70f9a, State: Running, Role: LEADER
I20260808 18:30:15.486732 29554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } }
I20260808 18:30:15.487413 29549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:30:15.487704 29554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8962b17c345d40fd9c9c39e1b5d70f9a. Latest consensus state: current_term: 3 leader_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } } }
I20260808 18:30:15.487828 29554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:15.488317 29555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8962b17c345d40fd9c9c39e1b5d70f9a" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45267 } } }
I20260808 18:30:15.488517 29555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8962b17c345d40fd9c9c39e1b5d70f9a [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:15.489537 29559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:30:15.495169 29559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:30:15.495844 29559 catalog_manager.cc:1259] Loaded cluster ID: 9b8a07200a6d40ad8672f352fbcab626
I20260808 18:30:15.495926 29559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 18:30:15.497061 29559 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f58520179ad google::LogMessage::Fail() at ??:0
@ 0x7f585201a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f58520174de google::LogMessage::Flush() at ??:0
@ 0x7f5852018a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f585c956fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f585c957d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f585c954579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f585c998640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f58542693d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5854269c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f585426b3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f585425a512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f585589e6db start_thread at ??:0
@ 0x7f5850b1561f clone at ??:0
W20260808 18:30:15.594122 29552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:15.594471 29552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:15.594550 29552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:15.598953 29552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:15.599036 29552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:15.599102 29552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:30:15.604996 29552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:46853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=43107
--tserver_master_addrs=127.7.164.126:45267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:15.606160 29552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:15.607262 29552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:15.617759 29602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:15.617780 29608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:15.618014 29552 server_base.cc:1034] running on GCE node
W20260808 18:30:15.617759 29603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:15.618683 29552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:15.619508 29552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:15.620711 29552 hybrid_clock.cc:648] HybridClock initialized: now 1786213815620687 us; error 33 us; skew 500 ppm
Aug 08 18:30:15 dist-test-slave-jztv krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213815, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:15.624415 29552 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:30:15.625658 29552 webserver.cc:460] Webserver started at http://127.7.164.65:43107/ using document root <none> and password file <none>
I20260808 18:30:15.626456 29552 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:15.626559 29552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:15.631883 29552 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 18:30:15.634558 29624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:15.636204 29552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:30:15.636343 29552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "c27b291221ae41158d078f84c3e2811e"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "732405baeac4196a708b3d57143b397c"
server_key_iv: "c8fd6a91464c6ebb7e2aec4188470df3"
server_key_version: "encryptionkey@0"
I20260808 18:30:15.637072 29552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:15.656699 29552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:15.660872 29552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:15.661109 29552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:15.661952 29552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:15.663445 29637 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 18:30:15.667906 29552 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 18:30:15.668001 29552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.001s
I20260808 18:30:15.668069 29552 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 18:30:15.670439 29552 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 18:30:15.670506 29552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.002s
I20260808 18:30:15.670819 29637 tablet_bootstrap.cc:492] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e: Bootstrap starting.
I20260808 18:30:15.681954 29552 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:46853
I20260808 18:30:15.682667 29804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:46853 every 8 connection(s)
I20260808 18:30:15.683283 29552 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:30:15.691684 29637 log.cc:824] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:15.691926 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 29552
I20260808 18:30:15.692381 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:41509
--local_ip_for_outbound_sockets=127.7.164.66
--tserver_master_addrs=127.7.164.126:45267
--webserver_port=40615
--webserver_interface=127.7.164.66
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:30:15.699638 29637 tablet_bootstrap.cc:492] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:30:15.700093 29637 tablet_bootstrap.cc:492] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e: Bootstrap complete.
I20260808 18:30:15.701772 29637 ts_tablet_manager.cc:1397] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e: Time spent bootstrapping tablet: real 0.031s user 0.018s sys 0.010s
I20260808 18:30:15.704788 29637 raft_consensus.cc:348] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:15.705255 29637 raft_consensus.cc:729] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c27b291221ae41158d078f84c3e2811e, State: Initialized, Role: FOLLOWER
I20260808 18:30:15.705797 29637 consensus_queue.cc:260] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:15.706862 29637 ts_tablet_manager.cc:1428] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260808 18:30:15.735786 29809 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:30:15.685729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:49271)
0808 18:30:15.686482 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:15.686554 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:15.687736 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:15.734954 (+ 47218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":169,"threads_started":1}
W20260808 18:30:15.736506 29805 heartbeater.cc:644] Failed to heartbeat to 127.7.164.126:45267 (0 consecutive failures): Network error: Failed to ping master at 127.7.164.126:45267: Client connection negotiation failed: client connection to 127.7.164.126:45267: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 18:30:15.737340 29809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.737060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:43177)
0808 18:30:15.737199 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:15.737279 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 18:30:15.738152 29809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.737889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:49475)
0808 18:30:15.738026 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:15.738084 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 18:30:15.738363 29805 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:30:15.824987 29811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:15.825268 29811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:15.825313 29811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:15.829352 29811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:15.829411 29811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:15.829469 29811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:30:15.834796 29811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:41509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=40615
--tserver_master_addrs=127.7.164.126:45267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:15.835778 29811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:15.836745 29811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:15.846637 29831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:15.846663 29824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:15.846637 29823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:15.846928 29811 server_base.cc:1034] running on GCE node
I20260808 18:30:15.847440 29811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:15.848071 29811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:15.849238 29811 hybrid_clock.cc:648] HybridClock initialized: now 1786213815849202 us; error 48 us; skew 500 ppm
Aug 08 18:30:15 dist-test-slave-jztv krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213815, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:15.852087 29811 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:30:15.852918 29811 webserver.cc:460] Webserver started at http://127.7.164.66:40615/ using document root <none> and password file <none>
I20260808 18:30:15.853441 29811 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:15.853499 29811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:15.857281 29811 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:30:15.859395 29843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:15.860639 29811 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:30:15.860759 29811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "bbe65ce30d3144c0a865e4f2bfa4162c"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "328fa7c30a4758b6fce308172c983704"
server_key_iv: "fa58761314c44a094881a986e5a78304"
server_key_version: "encryptionkey@0"
I20260808 18:30:15.861297 29811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:15.873813 29811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:15.877072 29811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:15.877295 29811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:15.877982 29811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:15.878933 29811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:15.878981 29811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:15.879045 29811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:15.879081 29811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:15.888886 29811 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:41509
I20260808 18:30:15.888911 29998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:41509 every 8 connection(s)
I20260808 18:30:15.889966 29811 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:30:15.893466 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 29811
I20260808 18:30:15.893989 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:39835
--local_ip_for_outbound_sockets=127.7.164.67
--tserver_master_addrs=127.7.164.126:45267
--webserver_port=35047
--webserver_interface=127.7.164.67
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:30:15.894541 30002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.892346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:55031)
0808 18:30:15.892839 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:15.893462 (+ 623us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":109,"threads_started":1}
W20260808 18:30:15.895071 30000 heartbeater.cc:644] Failed to heartbeat to 127.7.164.126:45267 (0 consecutive failures): Network error: Failed to ping master at 127.7.164.126:45267: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
I20260808 18:30:15.895835 30002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.895518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:41261)
0808 18:30:15.895675 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:15.895747 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 18:30:15.896512 30002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:15.896288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:59363)
0808 18:30:15.896407 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:15.896463 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 18:30:15.896699 30000 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:30:16.012152 30003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:16.012490 30003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:16.012538 30003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:16.016765 30003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:16.016841 30003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:16.016942 30003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:30:16.022558 30003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:39835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=35047
--tserver_master_addrs=127.7.164.126:45267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:16.023818 30003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:16.025213 30003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:16.035648 30010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:16.035648 30011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:16.035709 30013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:16.035866 30003 server_base.cc:1034] running on GCE node
I20260808 18:30:16.036387 30003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:16.037067 30003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:16.038295 30003 hybrid_clock.cc:648] HybridClock initialized: now 1786213816038257 us; error 51 us; skew 500 ppm
Aug 08 18:30:16 dist-test-slave-jztv krb5kdc[27902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213816, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:16.041225 30003 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:30:16.042256 30003 webserver.cc:460] Webserver started at http://127.7.164.67:35047/ using document root <none> and password file <none>
I20260808 18:30:16.043020 30003 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:16.043107 30003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:16.047216 30003 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:30:16.049381 30024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:16.050969 30003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:30:16.051076 30003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "d0d96fd4d43c4b25b304d804885f6bee"
format_stamp: "Formatted at 2026-08-08 18:30:13 on dist-test-slave-jztv"
server_key: "60052cddb8e7954e48c110504325204b"
server_key_iv: "9c37dfe00b6b257db78ddfcdc2d36d07"
server_key_version: "encryptionkey@0"
I20260808 18:30:16.051636 30003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:16.063405 30003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:16.066622 30003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:16.066783 30003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:16.067350 30003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:16.068156 30003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:16.068204 30003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:16.068243 30003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:16.068259 30003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:16.078182 30003 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:39835
I20260808 18:30:16.078205 30184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:39835 every 8 connection(s)
I20260808 18:30:16.079352 30003 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:30:16.082796 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 30003
I20260808 18:30:16.083596 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:16.081635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:40883)
0808 18:30:16.082445 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:16.082619 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":115,"threads_started":1}
W20260808 18:30:16.084105 30185 heartbeater.cc:644] Failed to heartbeat to 127.7.164.126:45267 (0 consecutive failures): Network error: Failed to ping master at 127.7.164.126:45267: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
I20260808 18:30:16.084774 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:16.084520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:50367)
0808 18:30:16.084665 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:16.084716 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 18:30:16.085444 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:16.085243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:45609)
0808 18:30:16.085354 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:16.085396 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 18:30:16.085636 30185 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 18:30:16.739456 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:16.738958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:46937)
0808 18:30:16.739261 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:16.739335 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":81,"threads_started":1}
I20260808 18:30:16.837374 30193 raft_consensus.cc:482] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:30:16.837661 30193 raft_consensus.cc:504] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:16.838881 30193 leader_election.cc:288] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835), bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509)
Aug 08 18:30:16 dist-test-slave-jztv krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213815, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.67@KRBTEST.COM
Aug 08 18:30:16 dist-test-slave-jztv krb5kdc[27902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213815, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.66@KRBTEST.COM
I20260808 18:30:16.856443 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:16.839168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:41509 (local address 127.7.164.65:54073)
0808 18:30:16.839649 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:16.839663 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:16.839756 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:16.841215 (+ 1459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:16.841221 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:16.841797 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:16.842313 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:16.842332 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:16.844005 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:16.844008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:16.844867 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:16.844877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:16.845594 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:16.845598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:16.845840 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:16.849559 (+ 3719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:16.849579 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:16.851683 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:16.853896 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:16.853899 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:16.853901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:16.854198 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:16.854618 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:16.854621 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:16.854623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:16.854701 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:16.855452 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:16.855462 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:16.855727 (+ 265us) client_negotiation.cc:769] Sending connection context
0808 18:30:16.855882 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 18:30:16.856205 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":269,"spinlock_wait_cycles":14592,"thread_start_us":63,"threads_started":1}
I20260808 18:30:16.856565 30195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:16.839355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:54073 (local address 127.7.164.66:41509)
0808 18:30:16.839735 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:16.839751 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:16.839812 (+ 61us) server_negotiation.cc:407] Connection header received
0808 18:30:16.840492 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:16.840498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:16.840844 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:16.841164 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:16.842506 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:16.843859 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:16.844991 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:16.845463 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:16.845639 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:16.851900 (+ 6261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:16.851905 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:16.851917 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:16.851948 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:16.853737 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:16.854362 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:16.854368 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:16.854373 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:16.854523 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:16.854825 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:16.854828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:16.854829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:16.855327 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:16.855442 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:16.856059 (+ 617us) server_negotiation.cc:299] Negotiation successful
0808 18:30:16.856397 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":103,"threads_started":1}
I20260808 18:30:16.856449 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:16.839168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:39835 (local address 127.7.164.65:34051)
0808 18:30:16.839296 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:16.839313 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:16.839445 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:16.841217 (+ 1772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:16.841222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:16.841773 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:16.842313 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:16.842331 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:16.844235 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:16.844239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:16.844867 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:16.844878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:16.845601 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:16.845608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:16.845838 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:16.849556 (+ 3718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:16.849573 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:16.851289 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:16.853710 (+ 2421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:16.853716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:16.853731 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:16.854024 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:16.854471 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:16.854475 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:16.854477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:16.854688 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:16.855519 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:16.855521 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:16.855728 (+ 207us) client_negotiation.cc:769] Sending connection context
0808 18:30:16.855920 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 18:30:16.856276 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":212,"spinlock_wait_cycles":31104}
I20260808 18:30:16.857040 30196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:16.839362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:34051 (local address 127.7.164.67:39835)
0808 18:30:16.839808 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:16.839819 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:16.839852 (+ 33us) server_negotiation.cc:407] Connection header received
0808 18:30:16.840492 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:16.840498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:16.840844 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:16.841165 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:16.842509 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:16.843882 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:16.844998 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:16.845463 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:16.845644 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:16.851571 (+ 5927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:16.851577 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:16.851590 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:16.851619 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:16.853541 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:16.854157 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:16.854164 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:16.854169 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:16.854360 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:16.854924 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:16.854927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:16.854928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:16.855421 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:16.855551 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:16.856549 (+ 998us) server_negotiation.cc:299] Negotiation successful
0808 18:30:16.856837 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":111,"threads_started":1}
W20260808 18:30:16.859269 29628 leader_election.cc:341] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509): Not found: Tablet not found: b8ecf88f4162411997521cca77a8df89
W20260808 18:30:16.859444 29627 leader_election.cc:341] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835): Not found: Tablet not found: b8ecf88f4162411997521cca77a8df89
I20260808 18:30:16.859505 29628 leader_election.cc:302] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c27b291221ae41158d078f84c3e2811e; no voters: bbe65ce30d3144c0a865e4f2bfa4162c, d0d96fd4d43c4b25b304d804885f6bee
I20260808 18:30:16.859819 30193 raft_consensus.cc:2726] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:30:16.897739 30197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:16.897303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:54957)
0808 18:30:16.897579 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:16.897641 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":92,"threads_started":1}
I20260808 18:30:17.086599 30198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:17.086175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:45589)
0808 18:30:17.086454 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:17.086506 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":121,"threads_started":1}
I20260808 18:30:17.740713 30199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:17.740266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:34333)
0808 18:30:17.740550 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:17.740610 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":82,"threads_started":1}
I20260808 18:30:17.899112 30202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:17.898641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:40801)
0808 18:30:17.898929 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:17.898993 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":80,"threads_started":1}
I20260808 18:30:18.087819 30203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:18.087339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:52811)
0808 18:30:18.087649 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:18.087709 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":83,"threads_started":1}
I20260808 18:30:18.742146 30206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:18.741688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:45199)
0808 18:30:18.741973 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:18.742046 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
I20260808 18:30:18.900386 30207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:18.899912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:44521)
0808 18:30:18.900207 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:18.900268 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":81,"threads_started":1}
I20260808 18:30:18.910672 30209 raft_consensus.cc:482] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:30:18.910810 30209 raft_consensus.cc:504] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:18.911219 30209 leader_election.cc:288] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835), bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509)
W20260808 18:30:18.912014 29628 leader_election.cc:341] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509): Not found: Tablet not found: b8ecf88f4162411997521cca77a8df89
W20260808 18:30:18.912111 29627 leader_election.cc:341] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835): Not found: Tablet not found: b8ecf88f4162411997521cca77a8df89
I20260808 18:30:18.912226 29627 leader_election.cc:302] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c27b291221ae41158d078f84c3e2811e; no voters: bbe65ce30d3144c0a865e4f2bfa4162c, d0d96fd4d43c4b25b304d804885f6bee
I20260808 18:30:18.912461 30209 raft_consensus.cc:2726] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:30:19.089107 30210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:19.088666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:35061)
0808 18:30:19.088935 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:19.088995 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":87,"threads_started":1}
I20260808 18:30:19.743573 30211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:19.743100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:53987)
0808 18:30:19.743392 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:19.743472 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":109,"threads_started":1}
I20260808 18:30:19.901696 30212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:19.901257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:58537)
0808 18:30:19.901530 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:19.901593 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":78,"threads_started":1}
I20260808 18:30:20.090459 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:20.089893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:33353)
0808 18:30:20.090276 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:20.090357 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":111,"threads_started":1}
I20260808 18:30:20.745044 30215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:20.744533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:35777)
0808 18:30:20.744867 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:20.744936 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":79,"threads_started":1}
I20260808 18:30:20.903191 30216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:20.902665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:54427)
0808 18:30:20.902982 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:20.903044 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":118,"threads_started":1}
I20260808 18:30:21.091655 30217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:21.091222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:39543)
0808 18:30:21.091504 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:21.091561 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":114,"threads_started":1}
I20260808 18:30:21.746389 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:21.745874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:49149)
0808 18:30:21.746142 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:21.746199 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":84,"threads_started":1}
I20260808 18:30:21.904503 30219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:21.904051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:34907)
0808 18:30:21.904336 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:21.904400 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":82,"threads_started":1}
I20260808 18:30:22.092935 30220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:22.092460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:58247)
0808 18:30:22.092766 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:22.092829 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":95,"threads_started":1}
I20260808 18:30:22.204177 30221 raft_consensus.cc:482] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:30:22.204370 30221 raft_consensus.cc:504] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:22.204916 30221 leader_election.cc:288] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835), bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509)
W20260808 18:30:22.205750 29627 leader_election.cc:341] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835): Not found: Tablet not found: b8ecf88f4162411997521cca77a8df89
W20260808 18:30:22.205899 29628 leader_election.cc:341] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509): Not found: Tablet not found: b8ecf88f4162411997521cca77a8df89
I20260808 18:30:22.205996 29628 leader_election.cc:302] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c27b291221ae41158d078f84c3e2811e; no voters: bbe65ce30d3144c0a865e4f2bfa4162c, d0d96fd4d43c4b25b304d804885f6bee
I20260808 18:30:22.206197 30221 raft_consensus.cc:2726] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:30:22.747681 30223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:22.747207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:32889)
0808 18:30:22.747520 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:22.747585 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":143,"threads_started":1}
I20260808 18:30:22.905821 30231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:22.905339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:44469)
0808 18:30:22.905662 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:22.905723 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":76,"threads_started":1}
I20260808 18:30:23.094440 30232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:23.093811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:38495)
0808 18:30:23.094170 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:23.094321 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":76,"threads_started":1}
I20260808 18:30:23.749095 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:23.748581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:50889)
0808 18:30:23.748928 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:23.748995 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":102,"threads_started":1}
I20260808 18:30:23.907127 30236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:23.906696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:46141)
0808 18:30:23.906967 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:23.907028 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":76,"threads_started":1}
I20260808 18:30:24.095722 30237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:24.095263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:45699)
0808 18:30:24.095537 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:24.095597 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":86,"threads_started":1}
I20260808 18:30:24.750618 30238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:24.750028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:55635)
0808 18:30:24.750439 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:24.750505 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":83,"threads_started":1}
I20260808 18:30:24.908399 30239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:24.907969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:47433)
0808 18:30:24.908238 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:24.908298 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":77,"threads_started":1}
I20260808 18:30:25.096913 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:25.096493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:45965)
0808 18:30:25.096743 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:25.096799 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
I20260808 18:30:25.574540 30241 raft_consensus.cc:482] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:30:25.574688 30241 raft_consensus.cc:504] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d96fd4d43c4b25b304d804885f6bee" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 39835 } } peers { permanent_uuid: "c27b291221ae41158d078f84c3e2811e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 46853 } } peers { permanent_uuid: "bbe65ce30d3144c0a865e4f2bfa4162c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41509 } }
I20260808 18:30:25.575261 30241 leader_election.cc:288] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835), bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509)
W20260808 18:30:25.576107 29628 leader_election.cc:341] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bbe65ce30d3144c0a865e4f2bfa4162c (127.7.164.66:41509): Not found: Tablet not found: b8ecf88f4162411997521cca77a8df89
W20260808 18:30:25.576200 29627 leader_election.cc:341] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0d96fd4d43c4b25b304d804885f6bee (127.7.164.67:39835): Not found: Tablet not found: b8ecf88f4162411997521cca77a8df89
I20260808 18:30:25.576315 29627 leader_election.cc:302] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c27b291221ae41158d078f84c3e2811e; no voters: bbe65ce30d3144c0a865e4f2bfa4162c, d0d96fd4d43c4b25b304d804885f6bee
I20260808 18:30:25.576522 30241 raft_consensus.cc:2726] T b8ecf88f4162411997521cca77a8df89 P c27b291221ae41158d078f84c3e2811e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:30:25.751962 30242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:25.751524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:51773)
0808 18:30:25.751781 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:25.751839 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":76,"threads_started":1}
I20260808 18:30:25.909613 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:25.909190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:44895)
0808 18:30:25.909456 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:25.909516 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":81,"threads_started":1}
I20260808 18:30:26.098179 30245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:26.097759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:58969)
0808 18:30:26.098027 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:26.098083 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":81,"threads_started":1}
I20260808 18:30:26.753365 30247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:26.752912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:49509)
0808 18:30:26.753206 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:26.753267 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":100,"threads_started":1}
I20260808 18:30:26.910840 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:26.910411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:41267)
0808 18:30:26.910685 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:26.910741 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":78,"threads_started":1}
I20260808 18:30:27.099468 30250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:27.099031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:59035)
0808 18:30:27.099304 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:27.099371 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":77,"threads_started":1}
I20260808 18:30:27.754590 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:27.754152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:44043)
0808 18:30:27.754423 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:27.754490 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":94,"threads_started":1}
I20260808 18:30:27.912151 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:27.911729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:41975)
0808 18:30:27.911994 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:27.912052 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":82,"threads_started":1}
I20260808 18:30:28.100710 30253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:28.100275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:57697)
0808 18:30:28.100556 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:28.100611 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":85,"threads_started":1}
I20260808 18:30:28.755823 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:28.755389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:36967)
0808 18:30:28.755667 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:28.755723 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":74,"threads_started":1}
I20260808 18:30:28.913540 30256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:28.913049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:35749)
0808 18:30:28.913353 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:28.913422 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":101,"threads_started":1}
I20260808 18:30:29.101949 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:29.101481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:51329)
0808 18:30:29.101733 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:29.101790 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
I20260808 18:30:29.757179 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:29.756720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:51583)
0808 18:30:29.757005 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:29.757073 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":82,"threads_started":1}
I20260808 18:30:29.914880 30260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:29.914392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:60919)
0808 18:30:29.914708 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:29.914770 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":87,"threads_started":1}
I20260808 18:30:30.103127 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:30.102704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:32911)
0808 18:30:30.102972 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:30.103028 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":82,"threads_started":1}
I20260808 18:30:30.758693 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:30.758208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.65:44531)
0808 18:30:30.758538 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:30.758596 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":84,"threads_started":1}
I20260808 18:30:30.916566 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:30.916106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.66:36727)
0808 18:30:30.916389 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:30.916454 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":91,"threads_started":1}
I20260808 18:30:31.093197 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 29552
I20260808 18:30:31.104697 30265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:31.104211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45267 (local address 127.7.164.67:44541)
0808 18:30:31.104523 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:31.104588 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":96,"threads_started":1}
I20260808 18:30:31.105856 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 29811
I20260808 18:30:31.119675 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 30003
2026-08-08T18:30:31Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19587 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:30:31 dist-test-slave-jztv krb5kdc[30270](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:30:31 dist-test-slave-jztv krb5kdc[30270](info): set up 2 sockets
Aug 08 18:30:31 dist-test-slave-jztv krb5kdc[30270](info): commencing operation
krb5kdc: starting...
W20260808 18:30:32.362140 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:30:32 dist-test-slave-jztv krb5kdc[30270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:30:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:30:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:30:32.497463 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:35521
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:35521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:32.617367 30287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:32.617671 30287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:32.617713 30287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:32.621918 30287 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 18:30:32.621999 30287 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 18:30:32.622015 30287 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 18:30:32.622030 30287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:30:32.622051 30287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:32.622069 30287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:32.622085 30287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:30:32.622100 30287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:30:32.627887 30287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:35521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:35521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:32.628970 30287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:32.629978 30287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:32.638718 30298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:32.638788 30301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:32.638994 30287 server_base.cc:1034] running on GCE node
W20260808 18:30:32.639086 30297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:32.639711 30287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:32.640916 30287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:32.642134 30287 hybrid_clock.cc:648] HybridClock initialized: now 1786213832642120 us; error 33 us; skew 500 ppm
Aug 08 18:30:32 dist-test-slave-jztv krb5kdc[30270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213832, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:32.645149 30287 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:30:32.646148 30287 webserver.cc:460] Webserver started at http://127.7.164.126:43327/ using document root <none> and password file <none>
I20260808 18:30:32.646760 30287 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:32.646860 30287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:32.647154 30287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:30:32.649335 30287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "66bb3d098f6b442786b2132d882b3e81"
format_stamp: "Formatted at 2026-08-08 18:30:32 on dist-test-slave-jztv"
server_key: "ba7b0963fcbb5a77816266ad74182aca"
server_key_iv: "75a8373f303e7aa4a611d4dab026a23b"
server_key_version: "encryptionkey@0"
I20260808 18:30:32.649919 30287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "66bb3d098f6b442786b2132d882b3e81"
format_stamp: "Formatted at 2026-08-08 18:30:32 on dist-test-slave-jztv"
server_key: "ba7b0963fcbb5a77816266ad74182aca"
server_key_iv: "75a8373f303e7aa4a611d4dab026a23b"
server_key_version: "encryptionkey@0"
I20260808 18:30:32.653868 30287 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 18:30:32.656525 30312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:32.657965 30287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:30:32.658143 30287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "66bb3d098f6b442786b2132d882b3e81"
format_stamp: "Formatted at 2026-08-08 18:30:32 on dist-test-slave-jztv"
server_key: "ba7b0963fcbb5a77816266ad74182aca"
server_key_iv: "75a8373f303e7aa4a611d4dab026a23b"
server_key_version: "encryptionkey@0"
I20260808 18:30:32.658309 30287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:32.682420 30287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:32.686084 30287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:32.686342 30287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:32.693943 30287 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:35521
I20260808 18:30:32.694458 30383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:35521 every 8 connection(s)
I20260808 18:30:32.695144 30287 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:30:32.696247 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 30287
I20260808 18:30:32.696375 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:30:32.696695 7825 external_mini_cluster.cc:1428] Setting key 90512349d691705dab484c875e3200e0
I20260808 18:30:32.698828 30385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:32.706710 30385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Bootstrap starting.
Aug 08 18:30:32 dist-test-slave-jztv krb5kdc[30270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:32.709767 30385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:32.710654 30385 log.cc:824] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:32.712670 30385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: No bootstrap required, opened a new log
I20260808 18:30:32.713809 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:32.698318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52850 (local address 127.7.164.126:35521)
0808 18:30:32.698766 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:32.698786 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:32.698834 (+ 48us) server_negotiation.cc:407] Connection header received
0808 18:30:32.699908 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:32.699935 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:32.700379 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:32.700780 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:32.702326 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:32.703898 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:32.704860 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:32.705604 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:32.705779 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:32.708795 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:32.708802 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:32.708819 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:32.708855 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:32.710848 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:32.711336 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:32.711342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:32.711346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:32.711471 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:32.711787 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:32.711790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:32.711792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:32.712212 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:32.712428 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:32.712746 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 18:30:32.713076 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":129,"threads_started":1}
I20260808 18:30:32.717016 30385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } }
I20260808 18:30:32.717272 30385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:32.717357 30385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66bb3d098f6b442786b2132d882b3e81, State: Initialized, Role: FOLLOWER
I20260808 18:30:32.717783 30385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } }
I20260808 18:30:32.717937 30385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:30:32.718026 30385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:30:32.718148 30385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:32.719125 30385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } }
I20260808 18:30:32.719542 30385 leader_election.cc:302] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66bb3d098f6b442786b2132d882b3e81; no voters:
I20260808 18:30:32.719885 30385 leader_election.cc:288] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:30:32.720027 30392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:30:32.720302 30392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 1 LEADER]: Becoming Leader. State: Replica: 66bb3d098f6b442786b2132d882b3e81, State: Running, Role: LEADER
I20260808 18:30:32.720700 30392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } }
I20260808 18:30:32.721303 30385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:30:32.722651 30394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66bb3d098f6b442786b2132d882b3e81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } } }
I20260808 18:30:32.722819 30394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:32.722667 30396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66bb3d098f6b442786b2132d882b3e81. Latest consensus state: current_term: 1 leader_uuid: "66bb3d098f6b442786b2132d882b3e81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } } }
I20260808 18:30:32.722927 30396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: This master's current role is: LEADER
W20260808 18:30:32.726330 30415 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:30:32.726428 30415 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:30:32.726516 30409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:30:32.727486 30409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:30:32.733541 30409 catalog_manager.cc:1347] Generated new cluster ID: c1af0600774548c8b4812e8c899f2da6
I20260808 18:30:32.733649 30409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:30:32.740864 30409 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:30:32.741858 30409 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:30:32.748644 30409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Generated new TSK 0
I20260808 18:30:32.749401 30409 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:30:32.786056 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35521
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:32.907426 30421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:32.907740 30421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:32.907819 30421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:32.911948 30421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:32.912051 30421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:32.912143 30421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:30:32.917565 30421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:32.918772 30421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:32.919850 30421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:30:32.930419 30421 server_base.cc:1034] running on GCE node
W20260808 18:30:32.930361 30428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:32.930361 30433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:32.930362 30429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:32.931087 30421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:32.931756 30421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:32.932971 30421 hybrid_clock.cc:648] HybridClock initialized: now 1786213832932934 us; error 48 us; skew 500 ppm
Aug 08 18:30:32 dist-test-slave-jztv krb5kdc[30270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213832, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:32.935923 30421 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:30:32.936859 30421 webserver.cc:460] Webserver started at http://127.7.164.65:38691/ using document root <none> and password file <none>
I20260808 18:30:32.937461 30421 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:32.937561 30421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:32.937829 30421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:30:32.940049 30421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "96955f9de78442039a187c1d83d19983"
format_stamp: "Formatted at 2026-08-08 18:30:32 on dist-test-slave-jztv"
server_key: "94e2ae97d3f8d01a55ed0ca2a1aced1e"
server_key_iv: "41544b5c4431f99ac0310617f9560441"
server_key_version: "encryptionkey@0"
I20260808 18:30:32.940649 30421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "96955f9de78442039a187c1d83d19983"
format_stamp: "Formatted at 2026-08-08 18:30:32 on dist-test-slave-jztv"
server_key: "94e2ae97d3f8d01a55ed0ca2a1aced1e"
server_key_iv: "41544b5c4431f99ac0310617f9560441"
server_key_version: "encryptionkey@0"
I20260808 18:30:32.944592 30421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 18:30:32.947170 30440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:32.948534 30421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:30:32.948678 30421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "96955f9de78442039a187c1d83d19983"
format_stamp: "Formatted at 2026-08-08 18:30:32 on dist-test-slave-jztv"
server_key: "94e2ae97d3f8d01a55ed0ca2a1aced1e"
server_key_iv: "41544b5c4431f99ac0310617f9560441"
server_key_version: "encryptionkey@0"
I20260808 18:30:32.948798 30421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:32.966014 30421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:32.969413 30421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:32.969673 30421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:32.970377 30421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:32.971279 30421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:32.971355 30421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:32.971431 30421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:32.971483 30421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:32.981149 30421 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:42393
I20260808 18:30:32.981176 30599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:42393 every 8 connection(s)
I20260808 18:30:32.982079 30421 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:30:32.984401 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 30421
I20260808 18:30:32.984542 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:30:32.984839 7825 external_mini_cluster.cc:1428] Setting key bec884bdf9d2fa307fc726888b86c734
Aug 08 18:30:32 dist-test-slave-jztv krb5kdc[30270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213832, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:30:32.997846 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:32.983982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:57523 (local address 127.7.164.126:35521)
0808 18:30:32.984173 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:32.984177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:32.985114 (+ 937us) server_negotiation.cc:407] Connection header received
0808 18:30:32.986701 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:32.986704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:32.986758 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:32.986856 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:32.988057 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:32.988862 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:32.990695 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:32.990935 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:32.991029 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:32.994008 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:32.994011 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:32.994013 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:32.994038 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:32.995589 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:32.996134 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:32.996138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:32.996140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:32.996259 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:32.996627 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:32.996631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:32.996633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:32.996831 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:32.996945 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:32.997527 (+ 582us) server_negotiation.cc:299] Negotiation successful
0808 18:30:32.997668 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 18:30:32.998806 30603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:32.984369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:57523)
0808 18:30:32.984927 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:32.984993 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:32.986495 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:32.987023 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:32.987030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:32.987403 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:32.987895 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:32.987909 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:32.989012 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:32.989018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:32.990551 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:32.990567 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:32.991110 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:32.991117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:32.991358 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:32.992157 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:32.992185 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:32.993816 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:32.995718 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:32.995725 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:32.995742 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:32.996024 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:32.996361 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:32.996365 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:32.996367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:32.996526 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:32.996945 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:32.996963 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:32.997263 (+ 300us) client_negotiation.cc:769] Sending connection context
0808 18:30:32.997463 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 18:30:32.997810 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":190,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
I20260808 18:30:33.000263 30601 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35521
I20260808 18:30:33.000525 30601 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:33.001048 30601 heartbeater.cc:511] Master 127.7.164.126:35521 requested a full tablet report, sending...
I20260808 18:30:33.002775 30331 ts_manager.cc:194] Registered new tserver with Master: 96955f9de78442039a187c1d83d19983 (127.7.164.65:42393)
I20260808 18:30:33.003952 30331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:57523
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:30:33.017469 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35521
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:33.131997 30606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:33.132314 30606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:33.132393 30606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:33.136448 30606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:33.136556 30606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:33.136653 30606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:30:33.141952 30606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:33.143065 30606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:33.144114 30606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:30:33.154047 30606 server_base.cc:1034] running on GCE node
W20260808 18:30:33.153999 30611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:33.154021 30612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:33.153999 30614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:33.154735 30606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:33.155396 30606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:33.156592 30606 hybrid_clock.cc:648] HybridClock initialized: now 1786213833156577 us; error 26 us; skew 500 ppm
Aug 08 18:30:33 dist-test-slave-jztv krb5kdc[30270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213833, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:33.159453 30606 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:30:33.160331 30606 webserver.cc:460] Webserver started at http://127.7.164.66:40877/ using document root <none> and password file <none>
I20260808 18:30:33.160854 30606 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:33.160913 30606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:33.161103 30606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:30:33.163230 30606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "8de88ea4590b436080ce672cda17ea0c"
format_stamp: "Formatted at 2026-08-08 18:30:33 on dist-test-slave-jztv"
server_key: "4f837b13e937b7a337efceaeded6e27b"
server_key_iv: "cb0e9c73828afec67f4c09d37e53d518"
server_key_version: "encryptionkey@0"
I20260808 18:30:33.163784 30606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "8de88ea4590b436080ce672cda17ea0c"
format_stamp: "Formatted at 2026-08-08 18:30:33 on dist-test-slave-jztv"
server_key: "4f837b13e937b7a337efceaeded6e27b"
server_key_iv: "cb0e9c73828afec67f4c09d37e53d518"
server_key_version: "encryptionkey@0"
I20260808 18:30:33.167613 30606 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:30:33.170120 30622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:33.171398 30606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:30:33.171514 30606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "8de88ea4590b436080ce672cda17ea0c"
format_stamp: "Formatted at 2026-08-08 18:30:33 on dist-test-slave-jztv"
server_key: "4f837b13e937b7a337efceaeded6e27b"
server_key_iv: "cb0e9c73828afec67f4c09d37e53d518"
server_key_version: "encryptionkey@0"
I20260808 18:30:33.171608 30606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:33.188992 30606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:33.192584 30606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:33.192761 30606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:33.193361 30606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:33.194330 30606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:33.194391 30606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:33.194435 30606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:33.194450 30606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:33.204381 30606 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:33373
I20260808 18:30:33.204401 30790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:33373 every 8 connection(s)
I20260808 18:30:33.205417 30606 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:30:33.215554 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 30606
I20260808 18:30:33.215687 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:30:33.216013 7825 external_mini_cluster.cc:1428] Setting key 65a95139c31d9d891dc5e484f4fcc851
Aug 08 18:30:33 dist-test-slave-jztv krb5kdc[30270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213833, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:33.221812 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.207490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:52387 (local address 127.7.164.126:35521)
0808 18:30:33.207663 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:33.207667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:33.208543 (+ 876us) server_negotiation.cc:407] Connection header received
0808 18:30:33.209971 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:33.209974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:33.210035 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:33.210121 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:33.211449 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.212026 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.213935 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.214164 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.214337 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.217494 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:33.217499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:33.217501 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:33.217530 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:33.219273 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:33.220004 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:33.220009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:33.220014 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:33.220118 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:33.220487 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:33.220493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:33.220498 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:33.220762 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:33.220874 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:33.221533 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 18:30:33.221675 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 18:30:33.223047 30793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.207882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:52387)
0808 18:30:33.208377 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:33.208426 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:33.209727 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:33.210340 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:33.210347 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:33.210751 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:33.211275 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:33.211290 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.212233 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.212239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:33.213782 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:33.213794 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.214420 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.214424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:33.214583 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.215307 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:33.215335 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:33.217284 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:33.219429 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:33.219439 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:33.219460 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:33.219858 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:33.220218 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:33.220220 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:33.220221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:33.220367 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:33.220881 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:33.220887 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:33.221203 (+ 316us) client_negotiation.cc:769] Sending connection context
0808 18:30:33.221471 (+ 268us) client_negotiation.cc:240] Negotiation successful
0808 18:30:33.221862 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":136,"threads_started":1}
I20260808 18:30:33.224522 30791 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35521
I20260808 18:30:33.224885 30791 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:33.225469 30791 heartbeater.cc:511] Master 127.7.164.126:35521 requested a full tablet report, sending...
I20260808 18:30:33.226675 30331 ts_manager.cc:194] Registered new tserver with Master: 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373)
I20260808 18:30:33.227267 30331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:52387
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:30:33.250533 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35521
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:33.369845 30796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:33.370131 30796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:33.370178 30796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:33.374413 30796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:33.374485 30796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:33.374588 30796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:30:33.380234 30796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:33.381561 30796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:33.382661 30796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:33.393019 30805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:33.393147 30796 server_base.cc:1034] running on GCE node
W20260808 18:30:33.393019 30807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:33.393164 30802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:33.393860 30796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:33.394593 30796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:33.395793 30796 hybrid_clock.cc:648] HybridClock initialized: now 1786213833395774 us; error 37 us; skew 500 ppm
Aug 08 18:30:33 dist-test-slave-jztv krb5kdc[30270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213833, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:33.398969 30796 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:30:33.399972 30796 webserver.cc:460] Webserver started at http://127.7.164.67:42877/ using document root <none> and password file <none>
I20260808 18:30:33.400574 30796 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:33.400640 30796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:33.400872 30796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:30:33.403317 30796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "5a86a64ef4ce428b95beb86e30b8214f"
format_stamp: "Formatted at 2026-08-08 18:30:33 on dist-test-slave-jztv"
server_key: "4268b1a8312b79e6103a7e3fbee39f00"
server_key_iv: "55d22f291d9a86137d642c143388c9cf"
server_key_version: "encryptionkey@0"
I20260808 18:30:33.403931 30796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "5a86a64ef4ce428b95beb86e30b8214f"
format_stamp: "Formatted at 2026-08-08 18:30:33 on dist-test-slave-jztv"
server_key: "4268b1a8312b79e6103a7e3fbee39f00"
server_key_iv: "55d22f291d9a86137d642c143388c9cf"
server_key_version: "encryptionkey@0"
I20260808 18:30:33.408028 30796 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:30:33.410694 30817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:33.412065 30796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:30:33.412235 30796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "5a86a64ef4ce428b95beb86e30b8214f"
format_stamp: "Formatted at 2026-08-08 18:30:33 on dist-test-slave-jztv"
server_key: "4268b1a8312b79e6103a7e3fbee39f00"
server_key_iv: "55d22f291d9a86137d642c143388c9cf"
server_key_version: "encryptionkey@0"
I20260808 18:30:33.412364 30796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:33.441732 30796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:33.445330 30796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:33.445552 30796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:33.446192 30796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:33.447134 30796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:33.447208 30796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:33.447280 30796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:33.447330 30796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:33.456848 30796 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:40407
I20260808 18:30:33.456869 30973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:40407 every 8 connection(s)
I20260808 18:30:33.457780 30796 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:30:33.459169 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 30796
I20260808 18:30:33.459319 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:30:33.459658 7825 external_mini_cluster.cc:1428] Setting key 68429b821b0153cc3a10541594c9b52a
Aug 08 18:30:33 dist-test-slave-jztv krb5kdc[30270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213833, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:33.472390 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.459828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:52761 (local address 127.7.164.126:35521)
0808 18:30:33.459989 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:33.459994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:33.461060 (+ 1066us) server_negotiation.cc:407] Connection header received
0808 18:30:33.462646 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:33.462649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:33.462694 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:33.462766 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:33.464040 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.464593 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.465713 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.465924 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.466007 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.468544 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:33.468547 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:33.468549 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:33.468574 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:33.470091 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:33.470685 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:33.470689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:33.470691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:33.470781 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:33.471156 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:33.471159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:33.471161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:33.471365 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:33.471497 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:33.472043 (+ 546us) server_negotiation.cc:299] Negotiation successful
0808 18:30:33.472208 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:30:33.473476 30977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.460312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:52761)
0808 18:30:33.460877 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:33.460950 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:33.462448 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:33.462973 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:33.462988 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:33.463382 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:33.463890 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:33.463904 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.464705 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.464708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:33.465593 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:33.465603 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.466060 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.466062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:33.466204 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.466795 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:33.466811 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:33.468351 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:33.470255 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:33.470262 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:33.470279 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:33.470573 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:33.470882 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:33.470886 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:33.470888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:33.471046 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:33.471497 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:33.471506 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:33.471781 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 18:30:33.471986 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 18:30:33.472347 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":174,"threads_started":1}
I20260808 18:30:33.474912 30974 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35521
I20260808 18:30:33.475172 30974 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:33.475723 30974 heartbeater.cc:511] Master 127.7.164.126:35521 requested a full tablet report, sending...
I20260808 18:30:33.476826 30331 ts_manager.cc:194] Registered new tserver with Master: 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67:40407)
I20260808 18:30:33.477414 30331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:52761
I20260808 18:30:33.486332 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:30:33.496106 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.488427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52856 (local address 127.7.164.126:35521)
0808 18:30:33.488539 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:33.488542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:33.488771 (+ 229us) server_negotiation.cc:407] Connection header received
0808 18:30:33.488867 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:33.488870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:33.488917 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:33.488982 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:33.489670 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.490272 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.491114 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.491349 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.491435 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.492849 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:33.492852 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:33.492854 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:33.492880 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:33.494487 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:33.494882 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:33.494885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:33.494886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:33.494971 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:33.495204 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:33.495206 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:33.495207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:33.495471 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:33.495580 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:33.495778 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 18:30:33.495881 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 18:30:33.499065 30331 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:30:33.502110 30331 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:30:33.516955 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.511004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55910 (local address 127.7.164.65:42393)
0808 18:30:33.511935 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:33.511943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:33.511970 (+ 27us) server_negotiation.cc:407] Connection header received
0808 18:30:33.512050 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:33.512056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:33.512275 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:33.512433 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:33.512738 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.513774 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.515318 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.516207 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.516317 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.516466 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:33.516681 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 18:30:33.516788 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":90,"threads_started":1}
I20260808 18:30:33.517223 30999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.511219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:40407 (local address 127.0.0.1:40108)
0808 18:30:33.511578 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:33.511591 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:33.511684 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:33.512973 (+ 1289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:33.512976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:33.512992 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:33.513115 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:33.513126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.514636 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.514640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:33.515760 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:33.515771 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.516814 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.516818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:33.516915 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.516937 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 18:30:33.516990 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 18:30:33.517058 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":101,"threads_started":1}
I20260808 18:30:33.516942 30996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.510820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:42393 (local address 127.0.0.1:55910)
0808 18:30:33.511222 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:33.511280 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:33.511427 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:33.512426 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:33.512432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:33.512455 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:33.512598 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:33.512607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.513892 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.513895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:33.515176 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:33.515188 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.516332 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.516336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:33.516459 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.516505 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 18:30:33.516629 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 18:30:33.516777 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":56,"thread_start_us":96,"threads_started":1}
I20260808 18:30:33.517599 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.511540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40108 (local address 127.7.164.67:40407)
0808 18:30:33.512520 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:33.512534 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:33.512558 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:30:33.512633 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:33.512638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:33.512826 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:33.512990 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:33.513252 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.514482 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.515906 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.516683 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.517148 (+ 465us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.517286 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:33.517341 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 18:30:33.517443 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":879,"thread_start_us":115,"threads_started":1}
I20260808 18:30:33.518303 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.512032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37160 (local address 127.7.164.66:33373)
0808 18:30:33.512748 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:33.512759 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:33.512785 (+ 26us) server_negotiation.cc:407] Connection header received
0808 18:30:33.512842 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:33.512846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:33.513026 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:33.513227 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:33.513483 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.514615 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.516094 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.516888 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.517801 (+ 913us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.517947 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:33.518029 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 18:30:33.518118 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":145,"threads_started":1}
I20260808 18:30:33.518582 31003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.511592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:33373 (local address 127.0.0.1:37160)
0808 18:30:33.512174 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:33.512190 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:33.512277 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:33.513232 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:33.513236 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:33.513256 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:33.513347 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:33.513353 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.514750 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.514757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:33.515858 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:33.515870 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.517191 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.517195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:33.517284 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.517319 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 18:30:33.517394 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 18:30:33.517464 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":24,"thread_start_us":100,"threads_started":1}
I20260808 18:30:33.520815 30886 tablet_service.cc:1467] Processing CreateTablet for tablet 3cee97ae938e48afa7e680ef6b620359 (DEFAULT_TABLE table=test-table [id=c0f8b482c816496db8dc21892d3e218c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:33.520815 30505 tablet_service.cc:1467] Processing CreateTablet for tablet 3cee97ae938e48afa7e680ef6b620359 (DEFAULT_TABLE table=test-table [id=c0f8b482c816496db8dc21892d3e218c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:33.521090 30697 tablet_service.cc:1467] Processing CreateTablet for tablet 3cee97ae938e48afa7e680ef6b620359 (DEFAULT_TABLE table=test-table [id=c0f8b482c816496db8dc21892d3e218c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:33.522040 30886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cee97ae938e48afa7e680ef6b620359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:33.522040 30505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cee97ae938e48afa7e680ef6b620359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:33.522233 30697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cee97ae938e48afa7e680ef6b620359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:33.528929 31008 tablet_bootstrap.cc:492] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c: Bootstrap starting.
I20260808 18:30:33.530172 31009 tablet_bootstrap.cc:492] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: Bootstrap starting.
I20260808 18:30:33.530385 31010 tablet_bootstrap.cc:492] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f: Bootstrap starting.
I20260808 18:30:33.532167 31008 tablet_bootstrap.cc:654] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:33.532434 31009 tablet_bootstrap.cc:654] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:33.532846 31010 tablet_bootstrap.cc:654] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:33.533039 31008 log.cc:824] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:33.533334 31009 log.cc:824] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:33.533629 31010 log.cc:824] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:33.535215 31009 tablet_bootstrap.cc:492] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: No bootstrap required, opened a new log
I20260808 18:30:33.535239 31008 tablet_bootstrap.cc:492] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c: No bootstrap required, opened a new log
I20260808 18:30:33.535439 31009 ts_tablet_manager.cc:1397] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:30:33.535444 31008 ts_tablet_manager.cc:1397] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 18:30:33.535537 31010 tablet_bootstrap.cc:492] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f: No bootstrap required, opened a new log
I20260808 18:30:33.535708 31010 ts_tablet_manager.cc:1397] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 18:30:33.539304 31009 raft_consensus.cc:348] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } }
I20260808 18:30:33.539605 31009 raft_consensus.cc:374] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:33.539470 31010 raft_consensus.cc:348] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } }
I20260808 18:30:33.539471 31008 raft_consensus.cc:348] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } }
I20260808 18:30:33.539682 31009 raft_consensus.cc:729] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96955f9de78442039a187c1d83d19983, State: Initialized, Role: FOLLOWER
I20260808 18:30:33.539752 31010 raft_consensus.cc:374] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:33.539752 31008 raft_consensus.cc:374] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:33.539813 31008 raft_consensus.cc:729] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8de88ea4590b436080ce672cda17ea0c, State: Initialized, Role: FOLLOWER
I20260808 18:30:33.539867 31010 raft_consensus.cc:729] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a86a64ef4ce428b95beb86e30b8214f, State: Initialized, Role: FOLLOWER
I20260808 18:30:33.540234 31009 consensus_queue.cc:260] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } }
I20260808 18:30:33.540309 31008 consensus_queue.cc:260] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } }
I20260808 18:30:33.540309 31010 consensus_queue.cc:260] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } }
I20260808 18:30:33.541260 30974 heartbeater.cc:503] Master 127.7.164.126:35521 was elected leader, sending a full tablet report...
I20260808 18:30:33.541321 31009 ts_tablet_manager.cc:1428] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:30:33.541320 31008 ts_tablet_manager.cc:1428] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:30:33.541321 31010 ts_tablet_manager.cc:1428] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260808 18:30:33.541669 30791 heartbeater.cc:503] Master 127.7.164.126:35521 was elected leader, sending a full tablet report...
I20260808 18:30:33.541669 30601 heartbeater.cc:503] Master 127.7.164.126:35521 was elected leader, sending a full tablet report...
I20260808 18:30:33.590655 31015 raft_consensus.cc:482] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:30:33.590903 31015 raft_consensus.cc:504] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } }
I20260808 18:30:33.592311 31015 leader_election.cc:288] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96955f9de78442039a187c1d83d19983 (127.7.164.65:42393), 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373)
I20260808 18:30:33.596930 30977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.592609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:42393 (local address 127.7.164.67:44701)
0808 18:30:33.592759 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:33.592776 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:33.592880 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:33.593249 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:33.593251 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:33.593270 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:33.593365 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:33.593369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.594574 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.594577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:33.595486 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:33.595495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.596420 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.596424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:33.596546 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.596576 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 18:30:33.596679 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 18:30:33.596772 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":1}
I20260808 18:30:33.597043 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.592727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:44701 (local address 127.7.164.65:42393)
0808 18:30:33.592887 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:33.592894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:33.592912 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:30:33.592981 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:33.592985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:33.593038 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:33.593123 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:33.593513 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.594447 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.595638 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.596279 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.596391 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.596441 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:33.596759 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 18:30:33.596874 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:30:33.597242 31017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.592939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:33373 (local address 127.7.164.67:56499)
0808 18:30:33.593439 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:33.593456 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:33.593579 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:33.593811 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:33.593815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:33.593838 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:33.593965 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:33.593972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.594879 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.594882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:33.595923 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:33.595935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.596773 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.596777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:33.596889 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.596913 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:30:33.597001 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 18:30:33.597088 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":179,"threads_started":1}
I20260808 18:30:33.597283 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.593047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:56499 (local address 127.7.164.66:33373)
0808 18:30:33.593216 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:33.593221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:33.593559 (+ 338us) server_negotiation.cc:407] Connection header received
0808 18:30:33.593664 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:33.593666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:33.593709 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:33.593768 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:33.594083 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.594747 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.596085 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.596634 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.596749 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.596795 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:33.597041 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 18:30:33.597133 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 18:30:33.598021 30532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cee97ae938e48afa7e680ef6b620359" candidate_uuid: "5a86a64ef4ce428b95beb86e30b8214f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96955f9de78442039a187c1d83d19983" is_pre_election: true
I20260808 18:30:33.598040 30723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cee97ae938e48afa7e680ef6b620359" candidate_uuid: "5a86a64ef4ce428b95beb86e30b8214f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8de88ea4590b436080ce672cda17ea0c" is_pre_election: true
I20260808 18:30:33.598387 30723 raft_consensus.cc:2454] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a86a64ef4ce428b95beb86e30b8214f in term 0.
I20260808 18:30:33.598389 30532 raft_consensus.cc:2454] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a86a64ef4ce428b95beb86e30b8214f in term 0.
I20260808 18:30:33.598922 30821 leader_election.cc:302] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a86a64ef4ce428b95beb86e30b8214f, 96955f9de78442039a187c1d83d19983; no voters:
I20260808 18:30:33.599247 31015 raft_consensus.cc:2781] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:30:33.599395 31015 raft_consensus.cc:482] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:30:33.599474 31015 raft_consensus.cc:3037] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:33.600589 31015 raft_consensus.cc:504] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } }
I20260808 18:30:33.600975 31015 leader_election.cc:288] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [CANDIDATE]: Term 1 election: Requested vote from peers 96955f9de78442039a187c1d83d19983 (127.7.164.65:42393), 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373)
I20260808 18:30:33.601507 30723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cee97ae938e48afa7e680ef6b620359" candidate_uuid: "5a86a64ef4ce428b95beb86e30b8214f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8de88ea4590b436080ce672cda17ea0c"
I20260808 18:30:33.601544 30532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cee97ae938e48afa7e680ef6b620359" candidate_uuid: "5a86a64ef4ce428b95beb86e30b8214f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96955f9de78442039a187c1d83d19983"
I20260808 18:30:33.601662 30723 raft_consensus.cc:3037] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:33.601697 30532 raft_consensus.cc:3037] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:33.602854 30723 raft_consensus.cc:2454] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a86a64ef4ce428b95beb86e30b8214f in term 1.
I20260808 18:30:33.602902 30532 raft_consensus.cc:2454] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a86a64ef4ce428b95beb86e30b8214f in term 1.
I20260808 18:30:33.603211 30821 leader_election.cc:302] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a86a64ef4ce428b95beb86e30b8214f, 8de88ea4590b436080ce672cda17ea0c; no voters:
I20260808 18:30:33.603457 31015 raft_consensus.cc:2781] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:30:33.603725 31015 raft_consensus.cc:686] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [term 1 LEADER]: Becoming Leader. State: Replica: 5a86a64ef4ce428b95beb86e30b8214f, State: Running, Role: LEADER
I20260808 18:30:33.604100 31015 consensus_queue.cc:237] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } }
I20260808 18:30:33.607695 30331 catalog_manager.cc:5495] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67). New cstate: current_term: 1 leader_uuid: "5a86a64ef4ce428b95beb86e30b8214f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } health_report { overall_health: HEALTHY } } }
I20260808 18:30:33.620065 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.616603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40124 (local address 127.7.164.67:40407)
0808 18:30:33.616746 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:33.616750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:33.616764 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:30:33.616932 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:33.616935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:33.616996 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:33.617088 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:30:33.617299 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.617804 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.618720 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:33.618934 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:33.619017 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:33.619275 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:30:33.619725 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:30:33.619819 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:33.619879 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 18:30:33.619929 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:30:33.626688 30532 raft_consensus.cc:1261] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Refusing update from remote peer 5a86a64ef4ce428b95beb86e30b8214f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:30:33.626688 30723 raft_consensus.cc:1261] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c [term 1 FOLLOWER]: Refusing update from remote peer 5a86a64ef4ce428b95beb86e30b8214f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:30:33.627466 31015 consensus_queue.cc:1035] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:30:33.627754 31018 consensus_queue.cc:1035] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [LEADER]: Connected to new peer: Peer: permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:30:33.636653 31025 mvcc.cc:205] Tried to move back new op lower bound from 7316331862526427136 to 7316331862445309952. Current Snapshot: MvccSnapshot[applied={T|T < 7316331862526427136}]
I20260808 18:30:33.646341 30331 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:30:33.646546 30331 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:30:33.649065 30331 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Sending DeleteTablet for 3 replicas of tablet 3cee97ae938e48afa7e680ef6b620359
I20260808 18:30:33.649899 30886 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cee97ae938e48afa7e680ef6b620359 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:33 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:40108
I20260808 18:30:33.649904 30697 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cee97ae938e48afa7e680ef6b620359 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:33 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:37160
I20260808 18:30:33.650091 30505 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cee97ae938e48afa7e680ef6b620359 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:33 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:55910
I20260808 18:30:33.650457 31038 tablet_replica.cc:330] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c: stopping tablet replica
I20260808 18:30:33.650591 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 30421
I20260808 18:30:33.654736 31039 tablet_replica.cc:330] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f: stopping tablet replica
I20260808 18:30:33.655184 31039 raft_consensus.cc:2229] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:30:33.655658 31039 raft_consensus.cc:2258] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:30:33.656747 31038 raft_consensus.cc:2229] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:30:33.657084 31038 raft_consensus.cc:2258] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:30:33.658715 31039 ts_tablet_manager.cc:1905] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:30:33.658717 31038 ts_tablet_manager.cc:1905] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:30:33.661614 31038 ts_tablet_manager.cc:1918] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:30:33.661852 31038 log.cc:1196] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/3cee97ae938e48afa7e680ef6b620359
I20260808 18:30:33.662312 31038 ts_tablet_manager.cc:1939] T 3cee97ae938e48afa7e680ef6b620359 P 8de88ea4590b436080ce672cda17ea0c: Deleting consensus metadata
I20260808 18:30:33.662401 31039 ts_tablet_manager.cc:1918] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:30:33.662581 31039 log.cc:1196] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/3cee97ae938e48afa7e680ef6b620359
I20260808 18:30:33.663055 31039 ts_tablet_manager.cc:1939] T 3cee97ae938e48afa7e680ef6b620359 P 5a86a64ef4ce428b95beb86e30b8214f: Deleting consensus metadata
I20260808 18:30:33.663939 30315 catalog_manager.cc:4841] TS 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373): tablet 3cee97ae938e48afa7e680ef6b620359 (table test-table [id=c0f8b482c816496db8dc21892d3e218c]) successfully deleted
I20260808 18:30:33.664292 30313 catalog_manager.cc:4841] TS 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67:40407): tablet 3cee97ae938e48afa7e680ef6b620359 (table test-table [id=c0f8b482c816496db8dc21892d3e218c]) successfully deleted
W20260808 18:30:33.664613 30315 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:42393 (error 108)
I20260808 18:30:33.665478 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 30606
I20260808 18:30:33.665740 31003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:33.665425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:42393 (local address 127.0.0.1:55922)
0808 18:30:33.665568 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:33.665643 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 18:30:33.674739 30315 catalog_manager.cc:4584] TS 96955f9de78442039a187c1d83d19983 (127.7.164.65:42393): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3cee97ae938e48afa7e680ef6b620359: Network error: Client connection negotiation failed: client connection to 127.7.164.65:42393: connect: Connection refused (error 111)
I20260808 18:30:33.678931 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 30796
I20260808 18:30:33.692628 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 30287
I20260808 18:30:53.704312 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:35521
--webserver_interface=127.7.164.126
--webserver_port=43327
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:35521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:30:53.822692 31042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:53.822999 31042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:53.823093 31042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:53.827167 31042 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 18:30:53.827243 31042 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 18:30:53.827272 31042 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 18:30:53.827304 31042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:30:53.827348 31042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:53.827411 31042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:53.827448 31042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:30:53.827482 31042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:30:53.832967 31042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:35521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:35521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=43327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:53.834131 31042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:53.835216 31042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:53.849618 31050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:53.849673 31042 server_base.cc:1034] running on GCE node
W20260808 18:30:53.849606 31049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:53.849609 31053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:53.850593 31042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:53.851749 31042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:53.852916 31042 hybrid_clock.cc:648] HybridClock initialized: now 1786213853852878 us; error 48 us; skew 500 ppm
Aug 08 18:30:53 dist-test-slave-jztv krb5kdc[30270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213853, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:53.855856 31042 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:30:53.856751 31042 webserver.cc:460] Webserver started at http://127.7.164.126:43327/ using document root <none> and password file <none>
I20260808 18:30:53.857307 31042 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:53.857380 31042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:53.861292 31042 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:30:53.863535 31062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:53.864851 31042 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:30:53.864991 31042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "66bb3d098f6b442786b2132d882b3e81"
format_stamp: "Formatted at 2026-08-08 18:30:32 on dist-test-slave-jztv"
server_key: "ba7b0963fcbb5a77816266ad74182aca"
server_key_iv: "75a8373f303e7aa4a611d4dab026a23b"
server_key_version: "encryptionkey@0"
I20260808 18:30:53.865607 31042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:53.877842 31042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:53.881088 31042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:53.881299 31042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:53.888640 31042 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:35521
I20260808 18:30:53.888922 31134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:35521 every 8 connection(s)
I20260808 18:30:53.889773 31042 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:30:53.892110 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 31042
I20260808 18:30:53.892604 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:42393
--local_ip_for_outbound_sockets=127.7.164.65
--tserver_master_addrs=127.7.164.126:35521
--webserver_port=38691
--webserver_interface=127.7.164.65
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:30:53.895046 31135 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:30:53.896751 31135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Bootstrap starting.
I20260808 18:30:53.915071 31135 log.cc:824] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:53.923957 31135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:30:53.924361 31135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Bootstrap complete.
I20260808 18:30:53.928682 31135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } }
I20260808 18:30:53.929078 31135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66bb3d098f6b442786b2132d882b3e81, State: Initialized, Role: FOLLOWER
I20260808 18:30:53.929558 31135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } }
I20260808 18:30:53.929674 31135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:30:53.929767 31135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:30:53.929883 31135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:30:53.931820 31135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } }
I20260808 18:30:53.932183 31135 leader_election.cc:302] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66bb3d098f6b442786b2132d882b3e81; no voters:
I20260808 18:30:53.932529 31135 leader_election.cc:288] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 18:30:53.932614 31140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 18:30:53.932861 31140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 2 LEADER]: Becoming Leader. State: Replica: 66bb3d098f6b442786b2132d882b3e81, State: Running, Role: LEADER
I20260808 18:30:53.933223 31140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } }
I20260808 18:30:53.933805 31135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:30:53.934918 31143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "66bb3d098f6b442786b2132d882b3e81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } } }
I20260808 18:30:53.935045 31143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:53.935381 31144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66bb3d098f6b442786b2132d882b3e81. Latest consensus state: current_term: 2 leader_uuid: "66bb3d098f6b442786b2132d882b3e81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } } }
I20260808 18:30:53.935475 31144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: This master's current role is: LEADER
I20260808 18:30:53.939599 31160 catalog_manager.cc:1259] Loaded cluster ID: c1af0600774548c8b4812e8c899f2da6
I20260808 18:30:53.939661 31160 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: loading cluster ID for follower catalog manager: success
I20260808 18:30:53.942198 31160 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: acquiring CA information for follower catalog manager: success
I20260808 18:30:53.943790 31160 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:30:53.943930 31148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:30:53.945837 31148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:30:53.946326 31148 catalog_manager.cc:1259] Loaded cluster ID: c1af0600774548c8b4812e8c899f2da6
I20260808 18:30:53.946385 31148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:30:53.947057 31148 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:30:53.947569 31148 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Loaded TSK: 0
I20260808 18:30:53.963091 31148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Generated new TSK 1
I20260808 18:30:53.963321 31148 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:30:54.009874 31138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:54.010162 31138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:54.010277 31138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:54.014341 31138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:54.014427 31138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:54.014516 31138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:30:54.019904 31138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:42393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=38691
--tserver_master_addrs=127.7.164.126:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:54.021037 31138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:54.022094 31138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:54.032914 31175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:54.032914 31171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:54.032909 31172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:54.033262 31138 server_base.cc:1034] running on GCE node
I20260808 18:30:54.033900 31138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:54.034866 31138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:54.036101 31138 hybrid_clock.cc:648] HybridClock initialized: now 1786213854036073 us; error 50 us; skew 500 ppm
Aug 08 18:30:54 dist-test-slave-jztv krb5kdc[30270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213854, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:54.039623 31138 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:30:54.040726 31138 webserver.cc:460] Webserver started at http://127.7.164.65:38691/ using document root <none> and password file <none>
I20260808 18:30:54.041446 31138 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:54.041559 31138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:54.046835 31138 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:30:54.049818 31184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:54.051390 31138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:30:54.051523 31138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "96955f9de78442039a187c1d83d19983"
format_stamp: "Formatted at 2026-08-08 18:30:32 on dist-test-slave-jztv"
server_key: "94e2ae97d3f8d01a55ed0ca2a1aced1e"
server_key_iv: "41544b5c4431f99ac0310617f9560441"
server_key_version: "encryptionkey@0"
I20260808 18:30:54.052273 31138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:54.086179 31138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:54.090411 31138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:54.090619 31138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:54.091358 31138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:54.092723 31192 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 18:30:54.097232 31138 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 18:30:54.097316 31138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.001s
I20260808 18:30:54.097381 31138 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 18:30:54.099383 31138 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 18:30:54.099457 31138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 18:30:54.099732 31192 tablet_bootstrap.cc:492] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: Bootstrap starting.
I20260808 18:30:54.109087 31138 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:42393
I20260808 18:30:54.109155 31351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:42393 every 8 connection(s)
I20260808 18:30:54.110039 31138 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:30:54.110913 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 31138
I20260808 18:30:54.111362 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:33373
--local_ip_for_outbound_sockets=127.7.164.66
--tserver_master_addrs=127.7.164.126:35521
--webserver_port=40877
--webserver_interface=127.7.164.66
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:30:54.120862 31192 log.cc:824] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:30:54 dist-test-slave-jztv krb5kdc[30270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213854, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:54.129313 31192 tablet_bootstrap.cc:492] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:30:54.129907 31192 tablet_bootstrap.cc:492] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: Bootstrap complete.
I20260808 18:30:54.131328 31355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.112435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:51657 (local address 127.7.164.126:35521)
0808 18:30:54.113005 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:54.113024 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:54.113537 (+ 513us) server_negotiation.cc:407] Connection header received
0808 18:30:54.115120 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:54.115133 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:54.115419 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:54.115690 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:54.117388 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.119186 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.120880 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.121763 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.122042 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.125252 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:54.125262 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:54.125283 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:54.125326 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:54.127571 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:54.128269 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:54.128275 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:54.128280 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:54.128436 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:54.128818 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:54.128821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:54.128823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:54.129283 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:54.129417 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:54.130104 (+ 687us) server_negotiation.cc:299] Negotiation successful
0808 18:30:54.130455 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":142,"threads_started":1}
I20260808 18:30:54.131374 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.112391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:51657)
0808 18:30:54.113316 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:54.113386 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:54.114670 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:54.115870 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:54.115880 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:54.116381 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:54.117140 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.117167 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.119371 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.119375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.120719 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.120732 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.121983 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.121987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.122195 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.122965 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:54.122986 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:54.125008 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:54.127788 (+ 2780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:54.127794 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:54.127807 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:54.128137 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:54.128540 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:54.128543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:54.128545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:54.128715 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:54.129422 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:54.129430 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:54.129745 (+ 315us) client_negotiation.cc:769] Sending connection context
0808 18:30:54.129965 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 18:30:54.130358 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":146,"threads_started":1}
I20260808 18:30:54.131942 31192 ts_tablet_manager.cc:1397] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: Time spent bootstrapping tablet: real 0.032s user 0.020s sys 0.011s
I20260808 18:30:54.134114 31352 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35521
I20260808 18:30:54.134382 31352 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:54.134981 31352 heartbeater.cc:511] Master 127.7.164.126:35521 requested a full tablet report, sending...
I20260808 18:30:54.135519 31192 raft_consensus.cc:348] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } }
I20260808 18:30:54.135995 31192 raft_consensus.cc:729] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96955f9de78442039a187c1d83d19983, State: Initialized, Role: FOLLOWER
I20260808 18:30:54.136498 31192 consensus_queue.cc:260] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } }
I20260808 18:30:54.137562 31090 ts_manager.cc:194] Registered new tserver with Master: 96955f9de78442039a187c1d83d19983 (127.7.164.65:42393)
I20260808 18:30:54.141104 31090 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:51657
I20260808 18:30:54.141477 31192 ts_tablet_manager.cc:1428] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260808 18:30:54.143919 31352 heartbeater.cc:503] Master 127.7.164.126:35521 was elected leader, sending a full tablet report...
Aug 08 18:30:54 dist-test-slave-jztv krb5kdc[30270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213853, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for kudu/127.7.164.65@KRBTEST.COM
I20260808 18:30:54.152237 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.141222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60758 (local address 127.7.164.65:42393)
0808 18:30:54.142158 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:54.142168 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:54.142189 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:30:54.142305 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:54.142325 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:54.142529 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:54.142676 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:54.143433 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.144419 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.145707 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.146034 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.146120 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.148622 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:54.148631 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:54.148641 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:54.148663 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:54.150169 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:54.150785 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:54.150789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:54.150790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:54.150869 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:54.151167 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:54.151169 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:54.151171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:54.151398 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:54.151499 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:54.151951 (+ 452us) server_negotiation.cc:299] Negotiation successful
0808 18:30:54.152089 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":818,"thread_start_us":103,"threads_started":1}
I20260808 18:30:54.152307 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.140515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:42393 (local address 127.0.0.1:60758)
0808 18:30:54.141722 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:54.141769 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:54.141901 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:54.142702 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:54.142708 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:54.143162 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:54.143299 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.143310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.144554 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.144557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.145592 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.145605 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.146151 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.146155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.146310 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.146864 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:54.146880 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:54.148488 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:54.150353 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:54.150364 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:54.150369 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:54.150669 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:54.150963 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:54.150966 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:54.150968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:54.151076 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:54.151513 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:54.151519 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:54.151660 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 18:30:54.151829 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 18:30:54.152049 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1029,"thread_start_us":124,"threads_started":1}
I20260808 18:30:54.153533 31252 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cee97ae938e48afa7e680ef6b620359 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:33 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:60758
I20260808 18:30:54.154480 31369 tablet_replica.cc:330] stopping tablet replica
I20260808 18:30:54.154791 31369 raft_consensus.cc:2229] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:30:54.155061 31369 raft_consensus.cc:2258] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:30:54.156061 31369 ts_tablet_manager.cc:1905] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:30:54.160166 31369 ts_tablet_manager.cc:1918] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:30:54.160256 31369 log.cc:1196] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal/wals/3cee97ae938e48afa7e680ef6b620359
I20260808 18:30:54.160575 31369 ts_tablet_manager.cc:1939] T 3cee97ae938e48afa7e680ef6b620359 P 96955f9de78442039a187c1d83d19983: Deleting consensus metadata
I20260808 18:30:54.162425 31065 catalog_manager.cc:4841] TS 96955f9de78442039a187c1d83d19983 (127.7.164.65:42393): tablet 3cee97ae938e48afa7e680ef6b620359 (table test-table [id=c0f8b482c816496db8dc21892d3e218c]) successfully deleted
W20260808 18:30:54.238435 31359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:54.238734 31359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:54.238824 31359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:54.243429 31359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:54.243511 31359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:54.243602 31359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:30:54.248977 31359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:33373
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=40877
--tserver_master_addrs=127.7.164.126:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:54.250124 31359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:54.251155 31359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:54.261126 31380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:54.261126 31378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:54.261139 31377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:54.261580 31359 server_base.cc:1034] running on GCE node
I20260808 18:30:54.261992 31359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:54.262708 31359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:54.263865 31359 hybrid_clock.cc:648] HybridClock initialized: now 1786213854263833 us; error 47 us; skew 500 ppm
Aug 08 18:30:54 dist-test-slave-jztv krb5kdc[30270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213854, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:54.266644 31359 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:30:54.267561 31359 webserver.cc:460] Webserver started at http://127.7.164.66:40877/ using document root <none> and password file <none>
I20260808 18:30:54.268124 31359 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:54.268203 31359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:54.272230 31359 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:30:54.274292 31391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:54.275447 31359 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:30:54.275597 31359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "8de88ea4590b436080ce672cda17ea0c"
format_stamp: "Formatted at 2026-08-08 18:30:33 on dist-test-slave-jztv"
server_key: "4f837b13e937b7a337efceaeded6e27b"
server_key_iv: "cb0e9c73828afec67f4c09d37e53d518"
server_key_version: "encryptionkey@0"
I20260808 18:30:54.276119 31359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:54.298128 31359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:54.301054 31359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:54.301257 31359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:54.301877 31359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:54.302743 31359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:54.302812 31359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:54.302883 31359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:54.302922 31359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:54.312062 31359 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:33373
I20260808 18:30:54.312104 31558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:33373 every 8 connection(s)
I20260808 18:30:54.313047 31359 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:30:54.321033 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 31359
I20260808 18:30:54.321602 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:40407
--local_ip_for_outbound_sockets=127.7.164.67
--tserver_master_addrs=127.7.164.126:35521
--webserver_port=42877
--webserver_interface=127.7.164.67
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:30:54 dist-test-slave-jztv krb5kdc[30270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213854, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:54.327148 31355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.314838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:59183 (local address 127.7.164.126:35521)
0808 18:30:54.315012 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:54.315016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:54.315801 (+ 785us) server_negotiation.cc:407] Connection header received
0808 18:30:54.317229 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:54.317232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:54.317282 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:54.317359 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:54.318571 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.319097 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.320223 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.320452 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.320525 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.323369 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:54.323377 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:54.323380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:54.323407 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:54.324875 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:54.325398 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:54.325402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:54.325403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:54.325498 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:54.325948 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:54.325950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:54.325951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:54.326130 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:54.326205 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:54.326911 (+ 706us) server_negotiation.cc:299] Negotiation successful
0808 18:30:54.327023 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 18:30:54.328456 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.315203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:59183)
0808 18:30:54.315650 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:54.315692 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:54.317025 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:54.317535 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:54.317541 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:54.317895 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:54.318428 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.318444 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.319218 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.319220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.320113 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.320124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.320577 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.320580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.320776 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.321461 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:54.321488 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:54.323172 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:54.324997 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:54.325003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:54.325019 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:54.325283 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:54.325629 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:54.325635 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:54.325640 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:54.325840 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:54.326267 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:54.326276 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:54.326629 (+ 353us) client_negotiation.cc:769] Sending connection context
0808 18:30:54.326877 (+ 248us) client_negotiation.cc:240] Negotiation successful
0808 18:30:54.327282 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":116,"threads_started":1}
I20260808 18:30:54.329715 31559 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35521
I20260808 18:30:54.330006 31559 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:54.330690 31559 heartbeater.cc:511] Master 127.7.164.126:35521 requested a full tablet report, sending...
I20260808 18:30:54.331897 31090 ts_manager.cc:194] Registered new tserver with Master: 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373)
I20260808 18:30:54.332474 31090 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:59183
W20260808 18:30:54.437530 31563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:30:54.437842 31563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:30:54.437927 31563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:30:54.442054 31563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:30:54.442142 31563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:30:54.442274 31563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:30:54.447680 31563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:40407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=42877
--tserver_master_addrs=127.7.164.126:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:30:54.448791 31563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:30:54.449796 31563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:30:54.459646 31569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:54.459687 31563 server_base.cc:1034] running on GCE node
W20260808 18:30:54.459789 31570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:30:54.459674 31573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:30:54.460286 31563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:30:54.460911 31563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:30:54.462101 31563 hybrid_clock.cc:648] HybridClock initialized: now 1786213854462083 us; error 27 us; skew 500 ppm
Aug 08 18:30:54 dist-test-slave-jztv krb5kdc[30270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213854, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:30:54.465020 31563 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:30:54.465896 31563 webserver.cc:460] Webserver started at http://127.7.164.67:42877/ using document root <none> and password file <none>
I20260808 18:30:54.466482 31563 fs_manager.cc:359] Metadata directory not provided
I20260808 18:30:54.466562 31563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:30:54.470669 31563 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:30:54.472711 31587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:54.473932 31563 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:30:54.474045 31563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "5a86a64ef4ce428b95beb86e30b8214f"
format_stamp: "Formatted at 2026-08-08 18:30:33 on dist-test-slave-jztv"
server_key: "4268b1a8312b79e6103a7e3fbee39f00"
server_key_iv: "55d22f291d9a86137d642c143388c9cf"
server_key_version: "encryptionkey@0"
I20260808 18:30:54.474642 31563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:30:54.504184 31563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:30:54.507180 31563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:30:54.507344 31563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:30:54.507905 31563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:30:54.508723 31563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:30:54.508771 31563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:54.508811 31563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:30:54.508833 31563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:30:54.518103 31563 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:40407
I20260808 18:30:54.518138 31741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:40407 every 8 connection(s)
I20260808 18:30:54.519150 31563 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:30:54.519724 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 31563
I20260808 18:30:54.529127 31355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.520521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46172 (local address 127.7.164.126:35521)
0808 18:30:54.520667 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:54.520673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:54.521138 (+ 465us) server_negotiation.cc:407] Connection header received
0808 18:30:54.521193 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:54.521196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:54.521299 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:54.521388 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:54.522367 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.522894 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.524088 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.524297 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.524373 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.525781 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:54.525783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:54.525785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:54.525807 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:54.527256 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:54.527729 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:54.527735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:54.527737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:54.527834 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:54.528162 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:54.528165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:54.528166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:54.528378 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:54.528540 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:54.528807 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 18:30:54.528926 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53,"spinlock_wait_cycles":896}
Aug 08 18:30:54 dist-test-slave-jztv krb5kdc[30270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213854, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:30:54.533322 31748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.520962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:58423 (local address 127.7.164.126:35521)
0808 18:30:54.521205 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:54.521208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:54.522045 (+ 837us) server_negotiation.cc:407] Connection header received
0808 18:30:54.523577 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:54.523580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:54.523620 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:54.523688 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:54.525119 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.525618 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.526823 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.527065 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.527169 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.529990 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:54.529993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:54.529995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:54.530016 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:54.531560 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:54.532015 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:54.532017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:54.532018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:54.532092 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:54.532402 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:54.532404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:54.532405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:54.532576 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:54.532652 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:54.533105 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 18:30:54.533200 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"spinlock_wait_cycles":768,"thread_start_us":102,"threads_started":1}
I20260808 18:30:54.534152 31749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.521359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:58423)
0808 18:30:54.521865 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:54.521926 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:54.523356 (+ 1430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:54.523863 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:54.523871 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:54.524298 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:30:54.524965 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.524986 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.525755 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.525758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.526694 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.526704 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.527210 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.527214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.527447 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.528033 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:30:54.528054 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:30:54.529836 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:30:54.531663 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:54.531668 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:54.531680 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:54.531929 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:54.532167 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:30:54.532169 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:30:54.532170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:30:54.532298 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:30:54.532663 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:30:54.532668 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:30:54.532893 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 18:30:54.533047 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 18:30:54.533286 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":115,"threads_started":1}
I20260808 18:30:54.535370 31745 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35521
I20260808 18:30:54.535611 31745 heartbeater.cc:464] Registering TS with master...
I20260808 18:30:54.536160 31745 heartbeater.cc:511] Master 127.7.164.126:35521 requested a full tablet report, sending...
I20260808 18:30:54.537151 31090 ts_manager.cc:194] Registered new tserver with Master: 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67:40407)
I20260808 18:30:54.537654 31090 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:58423
I20260808 18:30:54.542770 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:30:54.546625 31748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.543410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46180 (local address 127.7.164.126:35521)
0808 18:30:54.543524 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:54.543544 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:54.543732 (+ 188us) server_negotiation.cc:407] Connection header received
0808 18:30:54.543864 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:54.543867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:54.543957 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:54.544041 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:30:54.544253 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.544744 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.545597 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.545808 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.545880 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.546155 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:30:54.546323 (+ 168us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 18:30:54.546526 (+ 203us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46180: token expired
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 18:30:54.546705 31748 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46180: token expired
W20260808 18:30:54.547063 30978 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.164.126:35521)
I20260808 18:30:54.553768 31748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.547443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46194 (local address 127.7.164.126:35521)
0808 18:30:54.547562 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:54.547565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:54.547577 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:30:54.547673 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:54.547675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:54.547708 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:54.547769 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:30:54.548318 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.548788 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.549505 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.549705 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.549783 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.551052 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:30:54.551054 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:30:54.551056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:30:54.551078 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:30:54.552322 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:54.552710 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:54.552712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:54.552713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:54.552788 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:30:54.553013 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:30:54.553016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:30:54.553017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:30:54.553178 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:30:54.553287 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:54.553466 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 18:30:54.553554 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:30:54.556135 31090 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46194:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:30:54.559386 31090 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:30:54.567344 31252 tablet_service.cc:1467] Processing CreateTablet for tablet ed1a1d45d6b74deaa52d29d7ce374186 (DEFAULT_TABLE table=test-table [id=c9837d03b67247d3ac097c8c8a99c34a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:54.568035 31252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed1a1d45d6b74deaa52d29d7ce374186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:54.570756 31192 tablet_bootstrap.cc:492] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983: Bootstrap starting.
I20260808 18:30:54.571632 31192 tablet_bootstrap.cc:654] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:54.571847 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.565985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:40407 (local address 127.0.0.1:55562)
0808 18:30:54.566121 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:54.566152 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:54.566260 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:54.567368 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:54.567371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:54.567390 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:54.567570 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.567578 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.569198 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.569201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.570366 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.570377 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.571412 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.571415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.571512 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.571555 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 18:30:54.571641 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 18:30:54.571721 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":36}
I20260808 18:30:54.571892 31756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.566398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55562 (local address 127.7.164.67:40407)
0808 18:30:54.566967 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:54.566985 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:54.567004 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:30:54.567069 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:54.567073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:54.567229 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:54.567391 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:54.567706 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.569071 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.570501 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.571307 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.571413 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.571569 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:54.571671 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 18:30:54.571759 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":157,"threads_started":1}
I20260808 18:30:54.572049 31755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.566363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:33373 (local address 127.0.0.1:55568)
0808 18:30:54.566611 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:54.566625 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:54.566713 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:54.567447 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:54.567451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:54.567472 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:54.567647 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.567653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.568946 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.568950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.570331 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.570343 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.571593 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.571600 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.571707 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.571734 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:30:54.571796 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 18:30:54.571870 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":74,"thread_start_us":107,"threads_started":1}
I20260808 18:30:54.572738 31757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.566591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55568 (local address 127.7.164.66:33373)
0808 18:30:54.566931 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:54.566945 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:54.566971 (+ 26us) server_negotiation.cc:407] Connection header received
0808 18:30:54.567053 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:54.567059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:54.567251 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:54.567555 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:54.567742 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.568829 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.570464 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.571440 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.572234 (+ 794us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.572405 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:54.572462 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 18:30:54.572559 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":144,"threads_started":1}
I20260808 18:30:54.574868 31650 tablet_service.cc:1467] Processing CreateTablet for tablet ed1a1d45d6b74deaa52d29d7ce374186 (DEFAULT_TABLE table=test-table [id=c9837d03b67247d3ac097c8c8a99c34a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:54.575399 31462 tablet_service.cc:1467] Processing CreateTablet for tablet ed1a1d45d6b74deaa52d29d7ce374186 (DEFAULT_TABLE table=test-table [id=c9837d03b67247d3ac097c8c8a99c34a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:30:54.575970 31650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed1a1d45d6b74deaa52d29d7ce374186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:54.576375 31462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed1a1d45d6b74deaa52d29d7ce374186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:30:54.582911 31192 tablet_bootstrap.cc:492] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983: No bootstrap required, opened a new log
I20260808 18:30:54.583036 31192 ts_tablet_manager.cc:1397] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20260808 18:30:54.583546 31192 raft_consensus.cc:348] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:30:54.583675 31192 raft_consensus.cc:374] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:54.583729 31192 raft_consensus.cc:729] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96955f9de78442039a187c1d83d19983, State: Initialized, Role: FOLLOWER
I20260808 18:30:54.583854 31192 consensus_queue.cc:260] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:30:54.584146 31192 ts_tablet_manager.cc:1428] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:30:54.584800 31763 tablet_bootstrap.cc:492] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f: Bootstrap starting.
I20260808 18:30:54.585178 31762 tablet_bootstrap.cc:492] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c: Bootstrap starting.
I20260808 18:30:54.587821 31762 tablet_bootstrap.cc:654] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:54.587821 31763 tablet_bootstrap.cc:654] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f: Neither blocks nor log segments found. Creating new log.
I20260808 18:30:54.588761 31763 log.cc:824] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:54.588761 31762 log.cc:824] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:30:54.591023 31763 tablet_bootstrap.cc:492] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f: No bootstrap required, opened a new log
I20260808 18:30:54.591022 31762 tablet_bootstrap.cc:492] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c: No bootstrap required, opened a new log
I20260808 18:30:54.591233 31763 ts_tablet_manager.cc:1397] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 18:30:54.591233 31762 ts_tablet_manager.cc:1397] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260808 18:30:54.595266 31762 raft_consensus.cc:348] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:30:54.595266 31763 raft_consensus.cc:348] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:30:54.595570 31763 raft_consensus.cc:374] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:54.595630 31762 raft_consensus.cc:374] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:30:54.595670 31763 raft_consensus.cc:729] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a86a64ef4ce428b95beb86e30b8214f, State: Initialized, Role: FOLLOWER
I20260808 18:30:54.595693 31762 raft_consensus.cc:729] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8de88ea4590b436080ce672cda17ea0c, State: Initialized, Role: FOLLOWER
I20260808 18:30:54.596165 31763 consensus_queue.cc:260] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:30:54.596165 31762 consensus_queue.cc:260] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:30:54.596854 31745 heartbeater.cc:503] Master 127.7.164.126:35521 was elected leader, sending a full tablet report...
I20260808 18:30:54.597080 31559 heartbeater.cc:503] Master 127.7.164.126:35521 was elected leader, sending a full tablet report...
I20260808 18:30:54.597216 31763 ts_tablet_manager.cc:1428] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:30:54.597216 31762 ts_tablet_manager.cc:1428] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 18:30:54.612218 31353 tablet.cc:2369] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:30:54.770182 31746 tablet.cc:2369] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:30:54.803017 31769 raft_consensus.cc:482] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:30:54.803247 31769 raft_consensus.cc:504] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:30:54.804364 31769 leader_election.cc:288] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373), 96955f9de78442039a187c1d83d19983 (127.7.164.65:42393)
I20260808 18:30:54.807900 31749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.804584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:33373 (local address 127.7.164.67:51109)
0808 18:30:54.804710 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:54.804727 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:54.804821 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:54.805105 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:54.805108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:54.805127 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:54.805251 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.805256 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.805988 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.805993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.806820 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.806829 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.807468 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.807472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.807579 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.807597 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 18:30:54.807685 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 18:30:54.807758 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 18:30:54.807900 31757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.804686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:51109 (local address 127.7.164.66:33373)
0808 18:30:54.804824 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:54.804831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:54.804851 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:30:54.804916 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:54.804922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:54.804995 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:54.805093 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:54.805363 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.805870 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.806959 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.807338 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.807469 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.807525 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:54.807702 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 18:30:54.807771 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:30:54.808583 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.804871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:42393 (local address 127.7.164.67:55105)
0808 18:30:54.805285 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:54.805299 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:30:54.805419 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:30:54.805712 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:30:54.805715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:30:54.805732 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:30:54.805859 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.805864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.806624 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.806631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.807391 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:30:54.807400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.808174 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.808180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:30:54.808273 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.808300 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:30:54.808380 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 18:30:54.808448 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":141,"threads_started":1}
I20260808 18:30:54.808593 31483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1a1d45d6b74deaa52d29d7ce374186" candidate_uuid: "5a86a64ef4ce428b95beb86e30b8214f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8de88ea4590b436080ce672cda17ea0c" is_pre_election: true
I20260808 18:30:54.808640 31772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.805017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:55105 (local address 127.7.164.65:42393)
0808 18:30:54.805278 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:54.805283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:54.805380 (+ 97us) server_negotiation.cc:407] Connection header received
0808 18:30:54.805528 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:54.805531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:54.805587 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:54.805692 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:30:54.805966 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.806517 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.807516 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.807952 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.808122 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.808268 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:54.808416 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 18:30:54.808509 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":63,"threads_started":1}
I20260808 18:30:54.808979 31483 raft_consensus.cc:2454] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a86a64ef4ce428b95beb86e30b8214f in term 0.
I20260808 18:30:54.809248 31284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1a1d45d6b74deaa52d29d7ce374186" candidate_uuid: "5a86a64ef4ce428b95beb86e30b8214f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96955f9de78442039a187c1d83d19983" is_pre_election: true
I20260808 18:30:54.809509 31591 leader_election.cc:302] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a86a64ef4ce428b95beb86e30b8214f, 8de88ea4590b436080ce672cda17ea0c; no voters:
I20260808 18:30:54.809577 31284 raft_consensus.cc:2454] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a86a64ef4ce428b95beb86e30b8214f in term 0.
I20260808 18:30:54.809804 31769 raft_consensus.cc:2781] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:30:54.809916 31769 raft_consensus.cc:482] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:30:54.809975 31769 raft_consensus.cc:3037] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:54.811067 31769 raft_consensus.cc:504] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:30:54.811391 31769 leader_election.cc:288] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [CANDIDATE]: Term 1 election: Requested vote from peers 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373), 96955f9de78442039a187c1d83d19983 (127.7.164.65:42393)
I20260808 18:30:54.811743 31483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1a1d45d6b74deaa52d29d7ce374186" candidate_uuid: "5a86a64ef4ce428b95beb86e30b8214f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8de88ea4590b436080ce672cda17ea0c"
I20260808 18:30:54.811765 31284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1a1d45d6b74deaa52d29d7ce374186" candidate_uuid: "5a86a64ef4ce428b95beb86e30b8214f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96955f9de78442039a187c1d83d19983"
I20260808 18:30:54.811841 31483 raft_consensus.cc:3037] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:54.811895 31284 raft_consensus.cc:3037] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:30:54.812861 31483 raft_consensus.cc:2454] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a86a64ef4ce428b95beb86e30b8214f in term 1.
I20260808 18:30:54.812878 31284 raft_consensus.cc:2454] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a86a64ef4ce428b95beb86e30b8214f in term 1.
I20260808 18:30:54.813154 31591 leader_election.cc:302] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a86a64ef4ce428b95beb86e30b8214f, 8de88ea4590b436080ce672cda17ea0c; no voters:
I20260808 18:30:54.813354 31769 raft_consensus.cc:2781] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:30:54.813618 31769 raft_consensus.cc:686] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [term 1 LEADER]: Becoming Leader. State: Replica: 5a86a64ef4ce428b95beb86e30b8214f, State: Running, Role: LEADER
I20260808 18:30:54.813961 31769 consensus_queue.cc:237] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
W20260808 18:30:54.814109 31560 tablet.cc:2369] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:30:54.817251 31090 catalog_manager.cc:5495] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67). New cstate: current_term: 1 leader_uuid: "5a86a64ef4ce428b95beb86e30b8214f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } health_report { overall_health: UNKNOWN } } }
I20260808 18:30:54.864082 31756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.860482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55568 (local address 127.7.164.67:40407)
0808 18:30:54.860637 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 18:30:54.860641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:30:54.860656 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:30:54.860823 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:30:54.860827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:30:54.860884 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:30:54.860990 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:30:54.861245 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.861769 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.862599 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:30:54.862845 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:30:54.862949 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:30:54.863261 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:30:54.863675 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:30:54.863806 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 18:30:54.863869 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 18:30:54.863926 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:30:54.869921 31284 raft_consensus.cc:1261] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Refusing update from remote peer 5a86a64ef4ce428b95beb86e30b8214f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:30:54.869999 31483 raft_consensus.cc:1261] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c [term 1 FOLLOWER]: Refusing update from remote peer 5a86a64ef4ce428b95beb86e30b8214f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:30:54.870703 31773 consensus_queue.cc:1035] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [LEADER]: Connected to new peer: Peer: permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:30:54.870922 31769 consensus_queue.cc:1035] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:30:54.877494 31776 mvcc.cc:205] Tried to move back new op lower bound from 7316331949539328000 to 7316331949321011200. Current Snapshot: MvccSnapshot[applied={T|T < 7316331949539328000}]
I20260808 18:30:54.877871 31775 mvcc.cc:205] Tried to move back new op lower bound from 7316331949539328000 to 7316331949321011200. Current Snapshot: MvccSnapshot[applied={T|T < 7316331949539328000}]
I20260808 18:30:54.878036 31777 mvcc.cc:205] Tried to move back new op lower bound from 7316331949539328000 to 7316331949321011200. Current Snapshot: MvccSnapshot[applied={T|T < 7316331949539328000}]
I20260808 18:30:54.886759 31087 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:30:54.886906 31087 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:30:54.889523 31087 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Sending DeleteTablet for 3 replicas of tablet ed1a1d45d6b74deaa52d29d7ce374186
I20260808 18:30:54.890285 31252 tablet_service.cc:1514] Processing DeleteTablet for tablet ed1a1d45d6b74deaa52d29d7ce374186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:54 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:60758
I20260808 18:30:54.890413 31462 tablet_service.cc:1514] Processing DeleteTablet for tablet ed1a1d45d6b74deaa52d29d7ce374186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:54 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:55568
I20260808 18:30:54.890923 31650 tablet_service.cc:1514] Processing DeleteTablet for tablet ed1a1d45d6b74deaa52d29d7ce374186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:30:54 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:55562
I20260808 18:30:54.891086 31794 tablet_replica.cc:330] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c: stopping tablet replica
I20260808 18:30:54.891400 31794 raft_consensus.cc:2229] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:30:54.891582 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 31138
I20260808 18:30:54.900667 31794 raft_consensus.cc:2258] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:30:54.904125 31795 tablet_replica.cc:330] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f: stopping tablet replica
I20260808 18:30:54.904484 31795 raft_consensus.cc:2229] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:30:54.904911 31795 raft_consensus.cc:2258] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:30:54.906522 31795 ts_tablet_manager.cc:1905] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:30:54.909336 31795 ts_tablet_manager.cc:1918] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:30:54.909507 31795 log.cc:1196] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/ed1a1d45d6b74deaa52d29d7ce374186
I20260808 18:30:54.909927 31795 ts_tablet_manager.cc:1939] T ed1a1d45d6b74deaa52d29d7ce374186 P 5a86a64ef4ce428b95beb86e30b8214f: Deleting consensus metadata
I20260808 18:30:54.911295 31063 catalog_manager.cc:4841] TS 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67:40407): tablet ed1a1d45d6b74deaa52d29d7ce374186 (table test-table [id=c9837d03b67247d3ac097c8c8a99c34a]) successfully deleted
I20260808 18:30:54.911459 31794 ts_tablet_manager.cc:1905] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 18:30:54.912516 31065 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:42393 (error 108)
W20260808 18:30:54.913847 31755 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:30:54.913517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:42393 (local address 127.0.0.1:60774)
0808 18:30:54.913676 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:54.913752 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:42393: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 18:30:54.914036 31794 ts_tablet_manager.cc:1918] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:30:54.914141 31794 log.cc:1196] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/ed1a1d45d6b74deaa52d29d7ce374186
I20260808 18:30:54.914451 31794 ts_tablet_manager.cc:1939] T ed1a1d45d6b74deaa52d29d7ce374186 P 8de88ea4590b436080ce672cda17ea0c: Deleting consensus metadata
W20260808 18:30:54.913993 31065 catalog_manager.cc:4584] TS 96955f9de78442039a187c1d83d19983 (127.7.164.65:42393): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ed1a1d45d6b74deaa52d29d7ce374186: Network error: Client connection negotiation failed: client connection to 127.7.164.65:42393: connect: Connection reset by peer (error 104)
I20260808 18:30:54.914901 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 31359
I20260808 18:30:54.940430 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 31563
I20260808 18:30:54.940594 31755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:30:54.940280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:33373 (local address 127.0.0.1:55584)
0808 18:30:54.940428 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 18:30:54.940493 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.66:33373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 18:30:54.953634 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 31042
I20260808 18:31:14.969069 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:35521
--webserver_interface=127.7.164.126
--webserver_port=43327
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:35521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:15.088160 31800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:15.088459 31800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:15.088533 31800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:15.092535 31800 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 18:31:15.092617 31800 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 18:31:15.092644 31800 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 18:31:15.092669 31800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:15.092726 31800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:15.092787 31800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:15.092825 31800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:15.092859 31800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:15.098263 31800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:35521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:35521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=43327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:15.099319 31800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:15.100356 31800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:31:15.109176 31800 server_base.cc:1034] running on GCE node
W20260808 18:31:15.109158 31815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:15.109143 31809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:15.109326 31808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:15.109934 31800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:15.111047 31800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:15.112208 31800 hybrid_clock.cc:648] HybridClock initialized: now 1786213875112183 us; error 39 us; skew 500 ppm
Aug 08 18:31:15 dist-test-slave-jztv krb5kdc[30270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213875, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:15.114971 31800 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:31:15.115821 31800 webserver.cc:460] Webserver started at http://127.7.164.126:43327/ using document root <none> and password file <none>
I20260808 18:31:15.116374 31800 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:15.116493 31800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:15.120205 31800 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 18:31:15.122334 31822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:15.123481 31800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:31:15.123621 31800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "66bb3d098f6b442786b2132d882b3e81"
format_stamp: "Formatted at 2026-08-08 18:30:32 on dist-test-slave-jztv"
server_key: "ba7b0963fcbb5a77816266ad74182aca"
server_key_iv: "75a8373f303e7aa4a611d4dab026a23b"
server_key_version: "encryptionkey@0"
I20260808 18:31:15.124162 31800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:15.141908 31800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:15.145119 31800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:15.145346 31800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:15.152995 31800 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:35521
I20260808 18:31:15.153337 31894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:35521 every 8 connection(s)
I20260808 18:31:15.154098 31800 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:15.157838 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 31800
I20260808 18:31:15.158397 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:42393
--local_ip_for_outbound_sockets=127.7.164.65
--tserver_master_addrs=127.7.164.126:35521
--webserver_port=38691
--webserver_interface=127.7.164.65
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:31:15.158830 31895 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:31:15.161230 31895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Bootstrap starting.
I20260808 18:31:15.179411 31895 log.cc:824] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:15.190973 31895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:31:15.191371 31895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: Bootstrap complete.
I20260808 18:31:15.195605 31895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } }
I20260808 18:31:15.195989 31895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66bb3d098f6b442786b2132d882b3e81, State: Initialized, Role: FOLLOWER
I20260808 18:31:15.196475 31895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } }
I20260808 18:31:15.196599 31895 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:15.196686 31895 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:15.196810 31895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 2 FOLLOWER]: Advancing to term 3
I20260808 18:31:15.198771 31895 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } }
I20260808 18:31:15.199113 31895 leader_election.cc:302] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66bb3d098f6b442786b2132d882b3e81; no voters:
I20260808 18:31:15.199482 31895 leader_election.cc:288] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 18:31:15.199602 31899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 18:31:15.199810 31899 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [term 3 LEADER]: Becoming Leader. State: Replica: 66bb3d098f6b442786b2132d882b3e81, State: Running, Role: LEADER
I20260808 18:31:15.200182 31899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } }
I20260808 18:31:15.200742 31895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:15.201687 31900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "66bb3d098f6b442786b2132d882b3e81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } } }
I20260808 18:31:15.201810 31900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:15.201743 31903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66bb3d098f6b442786b2132d882b3e81. Latest consensus state: current_term: 3 leader_uuid: "66bb3d098f6b442786b2132d882b3e81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66bb3d098f6b442786b2132d882b3e81" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35521 } } }
I20260808 18:31:15.201862 31903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:15.206694 31922 catalog_manager.cc:1259] Loaded cluster ID: c1af0600774548c8b4812e8c899f2da6
I20260808 18:31:15.206761 31922 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: loading cluster ID for follower catalog manager: success
I20260808 18:31:15.209190 31922 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 66bb3d098f6b442786b2132d882b3e81: acquiring CA information for follower catalog manager: success
F20260808 18:31:15.210366 31922 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff60522f9ad google::LogMessage::Fail() at ??:0
@ 0x7ff60523288a google::LogMessage::SendToLog() at ??:0
@ 0x7ff60522f4de google::LogMessage::Flush() at ??:0
@ 0x7ff605230a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff607ef318d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff60fb94989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7ff60fb71023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7ff60fb7154b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7ff60fb6a652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7ff60fb699e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7ff60fbb0541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff607472512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff608ab66db start_thread at ??:0
@ 0x7ff603d2d61f clone at ??:0
W20260808 18:31:15.299432 31897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:15.299682 31897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:15.299729 31897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:15.303766 31897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:15.303820 31897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:15.303900 31897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:31:15.309114 31897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:42393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=38691
--tserver_master_addrs=127.7.164.126:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:15.310134 31897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:15.311156 31897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:15.323137 31954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:15.323130 31956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:15.323635 31953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:15.324347 31897 server_base.cc:1034] running on GCE node
I20260808 18:31:15.324872 31897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:15.325541 31897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:15.326732 31897 hybrid_clock.cc:648] HybridClock initialized: now 1786213875326703 us; error 47 us; skew 500 ppm
Aug 08 18:31:15 dist-test-slave-jztv krb5kdc[30270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213875, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:15.329619 31897 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:31:15.330718 31897 webserver.cc:460] Webserver started at http://127.7.164.65:38691/ using document root <none> and password file <none>
I20260808 18:31:15.331297 31897 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:15.331390 31897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:15.335419 31897 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:31:15.337677 31966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:15.339015 31897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:31:15.339150 31897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "96955f9de78442039a187c1d83d19983"
format_stamp: "Formatted at 2026-08-08 18:30:32 on dist-test-slave-jztv"
server_key: "94e2ae97d3f8d01a55ed0ca2a1aced1e"
server_key_iv: "41544b5c4431f99ac0310617f9560441"
server_key_version: "encryptionkey@0"
I20260808 18:31:15.339722 31897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:15.356393 31897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:15.359799 31897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:15.360021 31897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:15.360623 31897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:15.361788 31975 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 18:31:15.367413 31897 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 18:31:15.367511 31897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 18:31:15.367614 31897 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 18:31:15.369316 31897 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 18:31:15.369390 31897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 18:31:15.369699 31975 tablet_bootstrap.cc:492] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983: Bootstrap starting.
I20260808 18:31:15.380612 31897 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:42393
I20260808 18:31:15.380657 32128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:42393 every 8 connection(s)
I20260808 18:31:15.381695 31897 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:31:15.386731 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 31897
I20260808 18:31:15.387264 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:33373
--local_ip_for_outbound_sockets=127.7.164.66
--tserver_master_addrs=127.7.164.126:35521
--webserver_port=40877
--webserver_interface=127.7.164.66
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:31:15.393910 31975 log.cc:824] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:15.404492 31975 tablet_bootstrap.cc:492] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:31:15.404956 31975 tablet_bootstrap.cc:492] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983: Bootstrap complete.
I20260808 18:31:15.406533 31975 ts_tablet_manager.cc:1397] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983: Time spent bootstrapping tablet: real 0.037s user 0.025s sys 0.010s
I20260808 18:31:15.410014 31975 raft_consensus.cc:348] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:31:15.410600 31975 raft_consensus.cc:729] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96955f9de78442039a187c1d83d19983, State: Initialized, Role: FOLLOWER
I20260808 18:31:15.411203 31975 consensus_queue.cc:260] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:31:15.412295 31975 ts_tablet_manager.cc:1428] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260808 18:31:15.431967 32131 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:31:15.383641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:59453)
0808 18:31:15.384125 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:15.384178 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:15.385001 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:15.431206 (+ 46205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":148,"threads_started":1}
W20260808 18:31:15.432538 32129 heartbeater.cc:644] Failed to heartbeat to 127.7.164.126:35521 (0 consecutive failures): Network error: Failed to ping master at 127.7.164.126:35521: Client connection negotiation failed: client connection to 127.7.164.126:35521: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 18:31:15.433364 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:15.433093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:48863)
0808 18:31:15.433237 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:15.433303 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 18:31:15.434183 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:15.433976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:37315)
0808 18:31:15.434097 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:15.434139 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 18:31:15.434429 32129 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:31:15.514956 32133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:15.515242 32133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:15.515334 32133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:15.519877 32133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:15.519943 32133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:15.520044 32133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:31:15.525430 32133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:33373
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=40877
--tserver_master_addrs=127.7.164.126:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:15.526589 32133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:15.527633 32133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:15.537521 32143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:15.537542 32148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:15.537593 32133 server_base.cc:1034] running on GCE node
W20260808 18:31:15.537581 32141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:15.538300 32133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:15.538947 32133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:15.540145 32133 hybrid_clock.cc:648] HybridClock initialized: now 1786213875540107 us; error 53 us; skew 500 ppm
Aug 08 18:31:15 dist-test-slave-jztv krb5kdc[30270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213875, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:15.543184 32133 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:31:15.544142 32133 webserver.cc:460] Webserver started at http://127.7.164.66:40877/ using document root <none> and password file <none>
I20260808 18:31:15.544711 32133 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:15.544808 32133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:15.548974 32133 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:31:15.551116 32156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:15.552276 32133 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:31:15.552419 32133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "8de88ea4590b436080ce672cda17ea0c"
format_stamp: "Formatted at 2026-08-08 18:30:33 on dist-test-slave-jztv"
server_key: "4f837b13e937b7a337efceaeded6e27b"
server_key_iv: "cb0e9c73828afec67f4c09d37e53d518"
server_key_version: "encryptionkey@0"
I20260808 18:31:15.552970 32133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:15.572958 32133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:15.576381 32133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:15.576591 32133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:15.577195 32133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:15.578090 32133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:15.578162 32133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:15.578251 32133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:15.578306 32133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:15.587812 32133 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:33373
I20260808 18:31:15.587850 32317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:33373 every 8 connection(s)
I20260808 18:31:15.588809 32133 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:31:15.592675 32322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:15.590894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:43879)
0808 18:31:15.591356 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:15.591469 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":121,"threads_started":1}
W20260808 18:31:15.593165 32320 heartbeater.cc:644] Failed to heartbeat to 127.7.164.126:35521 (0 consecutive failures): Network error: Failed to ping master at 127.7.164.126:35521: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
I20260808 18:31:15.593801 32322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:15.593574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:56519)
0808 18:31:15.593690 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:15.593740 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 18:31:15.594400 32322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:15.594173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:58725)
0808 18:31:15.594297 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:15.594341 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 18:31:15.594575 32320 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 18:31:15.598045 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 32133
I20260808 18:31:15.598523 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:40407
--local_ip_for_outbound_sockets=127.7.164.67
--tserver_master_addrs=127.7.164.126:35521
--webserver_port=42877
--webserver_interface=127.7.164.67
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:15.711621 32323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:15.711895 32323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:15.711939 32323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:15.715982 32323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:15.716037 32323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:15.716094 32323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:31:15.721288 32323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:40407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=42877
--tserver_master_addrs=127.7.164.126:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:15.722369 32323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:15.723413 32323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:15.733304 32328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:15.733439 32323 server_base.cc:1034] running on GCE node
W20260808 18:31:15.733320 32329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:15.733323 32332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:15.734035 32323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:15.734766 32323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:15.735929 32323 hybrid_clock.cc:648] HybridClock initialized: now 1786213875735897 us; error 42 us; skew 500 ppm
Aug 08 18:31:15 dist-test-slave-jztv krb5kdc[30270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213875, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:15.738842 32323 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:31:15.739679 32323 webserver.cc:460] Webserver started at http://127.7.164.67:42877/ using document root <none> and password file <none>
I20260808 18:31:15.740201 32323 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:15.740259 32323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:15.744242 32323 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:31:15.746405 32343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:15.747674 32323 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:31:15.747799 32323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "5a86a64ef4ce428b95beb86e30b8214f"
format_stamp: "Formatted at 2026-08-08 18:30:33 on dist-test-slave-jztv"
server_key: "4268b1a8312b79e6103a7e3fbee39f00"
server_key_iv: "55d22f291d9a86137d642c143388c9cf"
server_key_version: "encryptionkey@0"
I20260808 18:31:15.748385 32323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:15.768348 32323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:15.771731 32323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:15.771878 32323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:15.772509 32323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:15.773335 32323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:15.773388 32323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:15.773473 32323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:15.773514 32323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:15.782564 32323 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:40407
I20260808 18:31:15.782598 32498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:40407 every 8 connection(s)
I20260808 18:31:15.783540 32323 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:31:15.786031 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 32323
I20260808 18:31:15.787757 32502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:15.785597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:36565)
0808 18:31:15.786104 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:15.786330 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":109,"threads_started":1}
W20260808 18:31:15.788247 32499 heartbeater.cc:644] Failed to heartbeat to 127.7.164.126:35521 (0 consecutive failures): Network error: Failed to ping master at 127.7.164.126:35521: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
I20260808 18:31:15.788846 32502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:15.788643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:39539)
0808 18:31:15.788746 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:15.788792 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 18:31:15.789465 32502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:15.789253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:41647)
0808 18:31:15.789360 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:15.789410 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 18:31:15.789670 32499 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 18:31:16.435447 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:16.435013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:35309)
0808 18:31:16.435281 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:16.435335 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":76,"threads_started":1}
I20260808 18:31:16.595517 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:16.595097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:33929)
0808 18:31:16.595338 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:16.595393 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":84,"threads_started":1}
I20260808 18:31:16.790657 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:16.790259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:45801)
0808 18:31:16.790501 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:16.790553 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":67,"threads_started":1}
I20260808 18:31:17.086192 32511 raft_consensus.cc:482] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:31:17.086457 32511 raft_consensus.cc:504] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:31:17.087662 32511 leader_election.cc:288] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67:40407), 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373)
Aug 08 18:31:17 dist-test-slave-jztv krb5kdc[30270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213875, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.67@KRBTEST.COM
Aug 08 18:31:17 dist-test-slave-jztv krb5kdc[30270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213875, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.66@KRBTEST.COM
I20260808 18:31:17.107225 32516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:17.088149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:37237 (local address 127.7.164.66:33373)
0808 18:31:17.088563 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:17.088582 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:17.088612 (+ 30us) server_negotiation.cc:407] Connection header received
0808 18:31:17.089539 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:17.089548 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:17.089997 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:17.090503 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:17.092134 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:17.093759 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:17.095229 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:17.095696 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:17.095875 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:17.102741 (+ 6866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:17.102765 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:17.102782 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:17.102818 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:17.104687 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:17.105204 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:17.105210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:17.105215 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:17.105334 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:17.105693 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:17.105697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:17.105698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:17.106087 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:17.106252 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:17.106793 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 18:31:17.107051 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":81,"threads_started":1}
I20260808 18:31:17.107218 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:17.088017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:33373 (local address 127.7.164.65:37237)
0808 18:31:17.088422 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:17.088436 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:17.088535 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:17.090492 (+ 1957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:17.090498 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:17.091089 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:17.091935 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:17.091958 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:17.093918 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:17.093922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:17.095098 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:17.095109 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:17.095826 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:17.095829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:17.096031 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:17.100468 (+ 4437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:31:17.100486 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:17.102548 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:17.104810 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:17.104814 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:17.104816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:17.105086 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:17.105440 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:17.105443 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:17.105445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:17.105592 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:17.106230 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:17.106238 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:17.106488 (+ 250us) client_negotiation.cc:769] Sending connection context
0808 18:31:17.106672 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 18:31:17.107009 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":271,"spinlock_wait_cycles":5504,"thread_start_us":70,"threads_started":1}
I20260808 18:31:17.107263 32512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:17.087838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:40407 (local address 127.7.164.65:51875)
0808 18:31:17.088186 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:17.088204 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:17.088313 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:17.090492 (+ 2179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:17.090498 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:17.091089 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:17.091936 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:17.091958 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:17.093918 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:17.093922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:17.095099 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:17.095111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:17.096034 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:17.096037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:17.096122 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:17.100447 (+ 4325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:31:17.100471 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:17.102188 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:17.104749 (+ 2561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:17.104756 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:17.104769 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:17.105086 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:17.105449 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:17.105453 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:17.105455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:17.105596 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:17.106210 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:17.106246 (+ 36us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:17.106488 (+ 242us) client_negotiation.cc:769] Sending connection context
0808 18:31:17.106671 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 18:31:17.107099 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":453,"spinlock_wait_cycles":16000,"thread_start_us":66,"threads_started":1}
I20260808 18:31:17.107265 32514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:17.088069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:51875 (local address 127.7.164.67:40407)
0808 18:31:17.088517 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:17.088543 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:17.088590 (+ 47us) server_negotiation.cc:407] Connection header received
0808 18:31:17.089539 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:17.089549 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:17.089997 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:17.090361 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:17.092318 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:17.093741 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:17.095234 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:17.095724 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:17.095892 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:17.102442 (+ 6550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:17.102447 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:17.102459 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:17.102487 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:17.104618 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:17.105204 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:17.105210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:17.105215 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:17.105344 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:17.105694 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:17.105699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:17.105700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:17.106087 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:17.106252 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:17.106786 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 18:31:17.107076 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":88,"threads_started":1}
W20260808 18:31:17.109752 31967 leader_election.cc:341] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67:40407): Not found: Tablet not found: ed1a1d45d6b74deaa52d29d7ce374186
W20260808 18:31:17.109958 31969 leader_election.cc:341] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373): Not found: Tablet not found: ed1a1d45d6b74deaa52d29d7ce374186
I20260808 18:31:17.110083 31969 leader_election.cc:302] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96955f9de78442039a187c1d83d19983; no voters: 5a86a64ef4ce428b95beb86e30b8214f, 8de88ea4590b436080ce672cda17ea0c
I20260808 18:31:17.110401 32511 raft_consensus.cc:2726] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:31:17.436705 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:17.436361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:51553)
0808 18:31:17.436533 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:17.436608 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 18:31:17.596691 32518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:17.596237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:54063)
0808 18:31:17.596531 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:17.596587 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":94,"threads_started":1}
I20260808 18:31:17.792013 32519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:17.791517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:58639)
0808 18:31:17.791837 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:17.791905 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":82,"threads_started":1}
I20260808 18:31:18.438297 32520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:18.437762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:40439)
0808 18:31:18.438093 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:18.438160 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":105,"threads_started":1}
I20260808 18:31:18.598063 32521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:18.597593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:52317)
0808 18:31:18.597901 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:18.597968 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":81,"threads_started":1}
I20260808 18:31:18.793455 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:18.792945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:56709)
0808 18:31:18.793276 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:18.793342 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":87,"threads_started":1}
I20260808 18:31:18.905352 32523 raft_consensus.cc:482] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:31:18.905517 32523 raft_consensus.cc:504] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:31:18.906020 32523 leader_election.cc:288] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67:40407), 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373)
W20260808 18:31:18.907059 31969 leader_election.cc:341] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373): Not found: Tablet not found: ed1a1d45d6b74deaa52d29d7ce374186
W20260808 18:31:18.907167 31967 leader_election.cc:341] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67:40407): Not found: Tablet not found: ed1a1d45d6b74deaa52d29d7ce374186
I20260808 18:31:18.907238 31967 leader_election.cc:302] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96955f9de78442039a187c1d83d19983; no voters: 5a86a64ef4ce428b95beb86e30b8214f, 8de88ea4590b436080ce672cda17ea0c
I20260808 18:31:18.907377 32523 raft_consensus.cc:2726] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:31:19.439585 32525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:19.439117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:54903)
0808 18:31:19.439423 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:19.439482 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":84,"threads_started":1}
I20260808 18:31:19.599458 32526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:19.598977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:39591)
0808 18:31:19.599267 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:19.599342 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":82,"threads_started":1}
I20260808 18:31:19.794863 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:19.794406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:42709)
0808 18:31:19.794691 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:19.794753 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":77,"threads_started":1}
I20260808 18:31:20.441041 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:20.440558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:45907)
0808 18:31:20.440871 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:20.440930 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":94,"threads_started":1}
I20260808 18:31:20.600803 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:20.600330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:56101)
0808 18:31:20.600636 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:20.600701 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":88,"threads_started":1}
I20260808 18:31:20.796157 32534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:20.795730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:45781)
0808 18:31:20.796004 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:20.796062 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":98,"threads_started":1}
I20260808 18:31:20.981657 32535 raft_consensus.cc:482] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:31:20.981804 32535 raft_consensus.cc:504] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:31:20.982353 32535 leader_election.cc:288] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67:40407), 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373)
W20260808 18:31:20.983213 31969 leader_election.cc:341] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373): Not found: Tablet not found: ed1a1d45d6b74deaa52d29d7ce374186
W20260808 18:31:20.983325 31967 leader_election.cc:341] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67:40407): Not found: Tablet not found: ed1a1d45d6b74deaa52d29d7ce374186
I20260808 18:31:20.983438 31967 leader_election.cc:302] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96955f9de78442039a187c1d83d19983; no voters: 5a86a64ef4ce428b95beb86e30b8214f, 8de88ea4590b436080ce672cda17ea0c
I20260808 18:31:20.983593 32535 raft_consensus.cc:2726] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:31:21.442448 32537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:21.441936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:36887)
0808 18:31:21.442263 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:21.442345 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":99,"threads_started":1}
I20260808 18:31:21.602192 32538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:21.601695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:54843)
0808 18:31:21.602013 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:21.602081 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":95,"threads_started":1}
I20260808 18:31:21.797484 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:21.797026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:35739)
0808 18:31:21.797317 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:21.797391 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":115,"threads_started":1}
I20260808 18:31:22.443815 32542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:22.443267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:42981)
0808 18:31:22.443597 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:22.443707 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":106,"threads_started":1}
I20260808 18:31:22.603576 32544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:22.603116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:46515)
0808 18:31:22.603422 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:22.603476 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":74,"threads_started":1}
I20260808 18:31:22.798921 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:22.798412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:44867)
0808 18:31:22.798748 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:22.798807 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":78,"threads_started":1}
I20260808 18:31:23.445055 32546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:23.444611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:58301)
0808 18:31:23.444893 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:23.444946 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":80,"threads_started":1}
I20260808 18:31:23.604897 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:23.604409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:40325)
0808 18:31:23.604731 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:23.604795 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":82,"threads_started":1}
I20260808 18:31:23.800117 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:23.799714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:40067)
0808 18:31:23.799967 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:23.800022 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
I20260808 18:31:24.446353 32550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:24.445849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:54943)
0808 18:31:24.446113 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:24.446176 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
I20260808 18:31:24.606369 32551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:24.605862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:39927)
0808 18:31:24.606138 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:24.606197 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":89,"threads_started":1}
I20260808 18:31:24.801433 32552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:24.800945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:60005)
0808 18:31:24.801254 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:24.801330 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":115,"threads_started":1}
I20260808 18:31:25.110596 32553 raft_consensus.cc:482] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:31:25.110736 32553 raft_consensus.cc:504] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:31:25.111232 32553 leader_election.cc:288] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67:40407), 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373)
W20260808 18:31:25.112106 31969 leader_election.cc:341] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373): Not found: Tablet not found: ed1a1d45d6b74deaa52d29d7ce374186
W20260808 18:31:25.112217 31967 leader_election.cc:341] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67:40407): Not found: Tablet not found: ed1a1d45d6b74deaa52d29d7ce374186
I20260808 18:31:25.112282 31967 leader_election.cc:302] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96955f9de78442039a187c1d83d19983; no voters: 5a86a64ef4ce428b95beb86e30b8214f, 8de88ea4590b436080ce672cda17ea0c
I20260808 18:31:25.112421 32553 raft_consensus.cc:2726] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:31:25.447950 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:25.447261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:36869)
0808 18:31:25.447689 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:25.447807 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":135,"threads_started":1}
I20260808 18:31:25.607753 32555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:25.607248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:39139)
0808 18:31:25.607582 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:25.607649 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":92,"threads_started":1}
I20260808 18:31:25.802668 32556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:25.802251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:36561)
0808 18:31:25.802509 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:25.802566 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":71,"threads_started":1}
I20260808 18:31:26.449306 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:26.448838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:55425)
0808 18:31:26.449140 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:26.449203 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":107,"threads_started":1}
I20260808 18:31:26.609055 32559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:26.608602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:52377)
0808 18:31:26.608892 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:26.608951 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":91,"threads_started":1}
I20260808 18:31:26.804005 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:26.803555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:59917)
0808 18:31:26.803825 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:26.803898 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":101,"threads_started":1}
I20260808 18:31:27.450634 32561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:27.450104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:46709)
0808 18:31:27.450465 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:27.450526 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":149,"threads_started":1}
I20260808 18:31:27.610704 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:27.610007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:52905)
0808 18:31:27.610476 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:27.610563 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":89,"threads_started":1}
I20260808 18:31:27.805470 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:27.804975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:45967)
0808 18:31:27.805292 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:27.805362 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":100,"threads_started":1}
I20260808 18:31:28.022603 32567 raft_consensus.cc:482] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:31:28.022744 32567 raft_consensus.cc:504] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a86a64ef4ce428b95beb86e30b8214f" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40407 } } peers { permanent_uuid: "8de88ea4590b436080ce672cda17ea0c" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 33373 } } peers { permanent_uuid: "96955f9de78442039a187c1d83d19983" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42393 } }
I20260808 18:31:28.023217 32567 leader_election.cc:288] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67:40407), 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373)
W20260808 18:31:28.024089 31967 leader_election.cc:341] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5a86a64ef4ce428b95beb86e30b8214f (127.7.164.67:40407): Not found: Tablet not found: ed1a1d45d6b74deaa52d29d7ce374186
W20260808 18:31:28.024188 31969 leader_election.cc:341] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8de88ea4590b436080ce672cda17ea0c (127.7.164.66:33373): Not found: Tablet not found: ed1a1d45d6b74deaa52d29d7ce374186
I20260808 18:31:28.024298 31969 leader_election.cc:302] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96955f9de78442039a187c1d83d19983; no voters: 5a86a64ef4ce428b95beb86e30b8214f, 8de88ea4590b436080ce672cda17ea0c
I20260808 18:31:28.024456 32567 raft_consensus.cc:2726] T ed1a1d45d6b74deaa52d29d7ce374186 P 96955f9de78442039a187c1d83d19983 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:31:28.452188 32569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:28.451641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:45875)
0808 18:31:28.451955 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:28.452044 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":116,"threads_started":1}
I20260808 18:31:28.612053 32570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:28.611603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:60505)
0808 18:31:28.611893 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:28.611954 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":81,"threads_started":1}
I20260808 18:31:28.806973 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:28.806515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:57149)
0808 18:31:28.806812 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:28.806876 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":103,"threads_started":1}
I20260808 18:31:29.453466 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:29.453038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:44331)
0808 18:31:29.453301 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:29.453357 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":84,"threads_started":1}
I20260808 18:31:29.613457 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:29.612950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:57833)
0808 18:31:29.613269 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:29.613339 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":94,"threads_started":1}
I20260808 18:31:29.808287 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:29.807840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:36919)
0808 18:31:29.808134 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:29.808189 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":81,"threads_started":1}
I20260808 18:31:30.454905 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:30.454464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.65:60079)
0808 18:31:30.454749 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:30.454806 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":96,"threads_started":1}
I20260808 18:31:30.615001 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:30.614538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.66:46491)
0808 18:31:30.614831 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:30.614896 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":81,"threads_started":1}
I20260808 18:31:30.792946 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 31897
I20260808 18:31:30.806124 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 32133
I20260808 18:31:30.811687 32578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:30.811235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35521 (local address 127.7.164.67:42535)
0808 18:31:30.811529 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:30.811589 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.126:35521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":87,"threads_started":1}
I20260808 18:31:30.816922 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 32323
2026-08-08T18:31:30Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59696 ms)
[----------] 30 tests from SecurityITest (250658 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:30 dist-test-slave-jztv krb5kdc[32582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:30 dist-test-slave-jztv krb5kdc[32582](info): set up 2 sockets
Aug 08 18:31:30 dist-test-slave-jztv krb5kdc[32582](info): commencing operation
krb5kdc: starting...
W20260808 18:31:32.066190 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:31:32 dist-test-slave-jztv krb5kdc[32582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:31:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:31:32.197296 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:46497
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:46497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:32.317232 32603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:32.317502 32603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:32.317545 32603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:32.321503 32603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:32.321566 32603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:32.321585 32603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:32.321602 32603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:32.321617 32603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:32.326963 32603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:46497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:46497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:32.328012 32603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:32.328984 32603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:32.337975 32609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:32.338029 32610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:32.338240 32603 server_base.cc:1034] running on GCE node
W20260808 18:31:32.338029 32612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:32.338914 32603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:32.340034 32603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:32.341223 32603 hybrid_clock.cc:648] HybridClock initialized: now 1786213892341172 us; error 39 us; skew 500 ppm
Aug 08 18:31:32 dist-test-slave-jztv krb5kdc[32582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213892, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:32.344245 32603 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:31:32.345266 32603 webserver.cc:460] Webserver started at http://127.7.164.126:35535/ using document root <none> and password file <none>
I20260808 18:31:32.345880 32603 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:32.345945 32603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:32.346182 32603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:32.348338 32603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "f1d772afd1274da1b4b03a88373ee14b"
format_stamp: "Formatted at 2026-08-08 18:31:32 on dist-test-slave-jztv"
server_key: "f5514ce859887a75eeeb0c2b5a414ea4"
server_key_iv: "e6acf3889e59919232ef574359a6d35a"
server_key_version: "encryptionkey@0"
I20260808 18:31:32.348956 32603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "f1d772afd1274da1b4b03a88373ee14b"
format_stamp: "Formatted at 2026-08-08 18:31:32 on dist-test-slave-jztv"
server_key: "f5514ce859887a75eeeb0c2b5a414ea4"
server_key_iv: "e6acf3889e59919232ef574359a6d35a"
server_key_version: "encryptionkey@0"
I20260808 18:31:32.352977 32603 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 18:31:32.355599 32622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:32.356969 32603 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:31:32.357143 32603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "f1d772afd1274da1b4b03a88373ee14b"
format_stamp: "Formatted at 2026-08-08 18:31:32 on dist-test-slave-jztv"
server_key: "f5514ce859887a75eeeb0c2b5a414ea4"
server_key_iv: "e6acf3889e59919232ef574359a6d35a"
server_key_version: "encryptionkey@0"
I20260808 18:31:32.357272 32603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:32.373844 32603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:32.377051 32603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:32.377278 32603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:32.385097 32603 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:46497
I20260808 18:31:32.385118 32693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:46497 every 8 connection(s)
I20260808 18:31:32.386099 32603 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:32.388958 32694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:32.394833 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 32603
I20260808 18:31:32.394910 32694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b: Bootstrap starting.
I20260808 18:31:32.394981 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:32.395323 7825 external_mini_cluster.cc:1428] Setting key df7b66c273a2505fc4c12601706b648e
I20260808 18:31:32.397722 32694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:32.398545 32694 log.cc:824] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:32.400451 32694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b: No bootstrap required, opened a new log
Aug 08 18:31:32 dist-test-slave-jztv krb5kdc[32582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:31:32.404331 32694 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d772afd1274da1b4b03a88373ee14b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 46497 } }
I20260808 18:31:32.404623 32694 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:32.404690 32694 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d772afd1274da1b4b03a88373ee14b, State: Initialized, Role: FOLLOWER
I20260808 18:31:32.405198 32694 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d772afd1274da1b4b03a88373ee14b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 46497 } }
I20260808 18:31:32.405344 32694 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:32.405440 32694 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:32.405555 32694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:32.406668 32694 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d772afd1274da1b4b03a88373ee14b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 46497 } }
I20260808 18:31:32.407027 32694 leader_election.cc:302] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1d772afd1274da1b4b03a88373ee14b; no voters:
I20260808 18:31:32.407406 32694 leader_election.cc:288] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:32.407534 32704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:32.407794 32704 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [term 1 LEADER]: Becoming Leader. State: Replica: f1d772afd1274da1b4b03a88373ee14b, State: Running, Role: LEADER
I20260808 18:31:32.408205 32704 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d772afd1274da1b4b03a88373ee14b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 46497 } }
I20260808 18:31:32.408854 32694 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:32.410069 32705 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1d772afd1274da1b4b03a88373ee14b. Latest consensus state: current_term: 1 leader_uuid: "f1d772afd1274da1b4b03a88373ee14b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d772afd1274da1b4b03a88373ee14b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 46497 } } }
I20260808 18:31:32.410192 32705 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:32.410637 32706 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1d772afd1274da1b4b03a88373ee14b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d772afd1274da1b4b03a88373ee14b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 46497 } } }
I20260808 18:31:32.410676 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:32.396819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60146 (local address 127.7.164.126:46497)
0808 18:31:32.397271 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:32.397298 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:32.397348 (+ 50us) server_negotiation.cc:407] Connection header received
0808 18:31:32.398076 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:32.398095 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:32.398494 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:32.398866 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:32.399713 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:32.400751 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:32.401615 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:32.402085 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:32.402288 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:32.405115 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:32.405121 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:32.405133 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:32.405160 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:32.407020 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:32.407558 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:32.407565 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:32.407572 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:32.407726 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:32.408081 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:32.408085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:32.408086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:32.408603 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:32.408765 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:32.409237 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 18:31:32.409561 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":137,"threads_started":1}
I20260808 18:31:32.411048 32718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:32.410801 32706 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:32.414902 32718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:32.420987 32718 catalog_manager.cc:1347] Generated new cluster ID: 477a9ccfb9054be5be47ac98da924ed0
I20260808 18:31:32.421068 32718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:32.426983 32718 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:32.427886 32718 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:32.434288 32718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b: Generated new TSK 0
I20260808 18:31:32.435124 32718 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:31:32.478132 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:46497
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:32.596005 32736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:32.596313 32736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:32.596392 32736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:32.600526 32736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:32.600631 32736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:32.600729 32736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:31:32.606328 32736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:46497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:32.607514 32736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:32.608655 32736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:32.619079 32746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:32.619222 32750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:32.619132 32736 server_base.cc:1034] running on GCE node
W20260808 18:31:32.619107 32747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:32.619853 32736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:32.620468 32736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:32.621652 32736 hybrid_clock.cc:648] HybridClock initialized: now 1786213892621629 us; error 25 us; skew 500 ppm
Aug 08 18:31:32 dist-test-slave-jztv krb5kdc[32582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213892, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:32.625115 32736 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:31:32.626072 32736 webserver.cc:460] Webserver started at http://127.7.164.65:41195/ using document root <none> and password file <none>
I20260808 18:31:32.626703 32736 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:32.626770 32736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:32.626950 32736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:32.629017 32736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "8356457ff32944c0bed9201ef4d968da"
format_stamp: "Formatted at 2026-08-08 18:31:32 on dist-test-slave-jztv"
server_key: "368aba3584bdb19be2f4e29387094a2b"
server_key_iv: "c21f6da8c1c1184f140a4b61e8cd0b7a"
server_key_version: "encryptionkey@0"
I20260808 18:31:32.629558 32736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "8356457ff32944c0bed9201ef4d968da"
format_stamp: "Formatted at 2026-08-08 18:31:32 on dist-test-slave-jztv"
server_key: "368aba3584bdb19be2f4e29387094a2b"
server_key_iv: "c21f6da8c1c1184f140a4b61e8cd0b7a"
server_key_version: "encryptionkey@0"
I20260808 18:31:32.633406 32736 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:31:32.636106 32760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:32.637420 32736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:31:32.637526 32736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "8356457ff32944c0bed9201ef4d968da"
format_stamp: "Formatted at 2026-08-08 18:31:32 on dist-test-slave-jztv"
server_key: "368aba3584bdb19be2f4e29387094a2b"
server_key_iv: "c21f6da8c1c1184f140a4b61e8cd0b7a"
server_key_version: "encryptionkey@0"
I20260808 18:31:32.637619 32736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:32.650115 32736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:32.653352 32736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:32.653528 32736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:32.654071 32736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:32.655002 32736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:32.655052 32736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:32.655092 32736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:32.655108 32736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:32.664495 32736 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:38227
I20260808 18:31:32.664577 458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:38227 every 8 connection(s)
I20260808 18:31:32.665462 32736 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:31:32.666164 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 32736
I20260808 18:31:32.666355 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:31:32.666683 7825 external_mini_cluster.cc:1428] Setting key 1ca0901fae979bb1c8dec8b9ad236001
Aug 08 18:31:32 dist-test-slave-jztv krb5kdc[32582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213892, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:31:32.680058 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:32.667487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:32837 (local address 127.7.164.126:46497)
0808 18:31:32.668106 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:32.668110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:32.668713 (+ 603us) server_negotiation.cc:407] Connection header received
0808 18:31:32.670101 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:32.670105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:32.670160 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:32.670258 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:32.671489 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:32.672034 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:32.673213 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:32.673468 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:32.673582 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:32.676203 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:32.676207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:32.676210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:32.676241 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:32.677886 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:32.678498 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:32.678501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:32.678503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:32.678585 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:32.678898 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:32.678900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:32.678901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:32.679086 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:32.679163 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:32.679736 (+ 573us) server_negotiation.cc:299] Negotiation successful
0808 18:31:32.679844 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
I20260808 18:31:32.680964 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:32.668015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:46497 (local address 127.7.164.65:32837)
0808 18:31:32.668561 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:32.668605 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:32.669877 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:32.670444 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:32.670451 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:32.670822 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:32.671316 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:32.671330 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:32.672173 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:32.672179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:32.673074 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:32.673085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:32.673639 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:32.673643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:32.673817 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:32.674487 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:31:32.674505 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:32.676013 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:32.678001 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:32.678006 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:32.678019 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:32.678384 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:32.678671 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:32.678673 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:32.678674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:32.678815 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:32.679177 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:32.679182 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:32.679514 (+ 332us) client_negotiation.cc:769] Sending connection context
0808 18:31:32.679684 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 18:31:32.679974 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":165,"threads_started":1}
I20260808 18:31:32.682396 459 heartbeater.cc:347] Connected to a master server at 127.7.164.126:46497
I20260808 18:31:32.682678 459 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:32.683188 459 heartbeater.cc:511] Master 127.7.164.126:46497 requested a full tablet report, sending...
I20260808 18:31:32.684924 32643 ts_manager.cc:194] Registered new tserver with Master: 8356457ff32944c0bed9201ef4d968da (127.7.164.65:38227)
I20260808 18:31:32.686735 32643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:32837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:31:32.697659 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:46497
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:32.813663 466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:32.813964 466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:32.814011 466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:32.818463 466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:32.818534 466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:32.818596 466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:31:32.824111 466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:46497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:32.825265 466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:32.826306 466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:32.836297 477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:32.836231 474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:32.836266 479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:32.836674 466 server_base.cc:1034] running on GCE node
I20260808 18:31:32.837106 466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:32.837837 466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:32.839030 466 hybrid_clock.cc:648] HybridClock initialized: now 1786213892839013 us; error 29 us; skew 500 ppm
Aug 08 18:31:32 dist-test-slave-jztv krb5kdc[32582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213892, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:32.841962 466 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:31:32.842927 466 webserver.cc:460] Webserver started at http://127.7.164.66:43343/ using document root <none> and password file <none>
I20260808 18:31:32.843515 466 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:32.843578 466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:32.843847 466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:32.845970 466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "aec8c3050caa4d2fb56d01084555f82d"
format_stamp: "Formatted at 2026-08-08 18:31:32 on dist-test-slave-jztv"
server_key: "4f907d374d8880173bd4b164f215a4a2"
server_key_iv: "bfc1b5d61891968ea92d7c0b31da0d31"
server_key_version: "encryptionkey@0"
I20260808 18:31:32.846617 466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "aec8c3050caa4d2fb56d01084555f82d"
format_stamp: "Formatted at 2026-08-08 18:31:32 on dist-test-slave-jztv"
server_key: "4f907d374d8880173bd4b164f215a4a2"
server_key_iv: "bfc1b5d61891968ea92d7c0b31da0d31"
server_key_version: "encryptionkey@0"
I20260808 18:31:32.850605 466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:31:32.853147 490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:32.854452 466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:31:32.854591 466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "aec8c3050caa4d2fb56d01084555f82d"
format_stamp: "Formatted at 2026-08-08 18:31:32 on dist-test-slave-jztv"
server_key: "4f907d374d8880173bd4b164f215a4a2"
server_key_iv: "bfc1b5d61891968ea92d7c0b31da0d31"
server_key_version: "encryptionkey@0"
I20260808 18:31:32.854712 466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:32.876295 466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:32.880363 466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:32.880616 466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:32.881330 466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:32.882350 466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:32.882441 466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:32.882524 466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:32.882575 466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:32.892912 466 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:45275
I20260808 18:31:32.892988 658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:45275 every 8 connection(s)
I20260808 18:31:32.893972 466 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:31:32.896111 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 466
I20260808 18:31:32.896440 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:31:32.896912 7825 external_mini_cluster.cc:1428] Setting key 65ba571d67a2aa3d11fe9b4ed83f8e88
Aug 08 18:31:32 dist-test-slave-jztv krb5kdc[32582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213892, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
I20260808 18:31:32.913801 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:32.896983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:40295 (local address 127.7.164.126:46497)
0808 18:31:32.897150 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:32.897157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:32.898243 (+ 1086us) server_negotiation.cc:407] Connection header received
0808 18:31:32.899953 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:32.899956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:32.900030 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:32.900121 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:32.901653 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:32.902348 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:32.903608 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:32.904055 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:32.904211 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:32.907952 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:32.907957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:32.907962 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:32.908007 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:32.910661 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:32.911404 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:32.911407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:32.911408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:32.911526 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:32.912152 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:32.912155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:32.912157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:32.912515 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:32.912670 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:32.913358 (+ 688us) server_negotiation.cc:299] Negotiation successful
0808 18:31:32.913527 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 18:31:32.914678 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:32.897506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:46497 (local address 127.7.164.66:40295)
0808 18:31:32.898050 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:32.898102 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:32.899716 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:32.900345 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:32.900353 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:32.900757 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:32.901430 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:32.901454 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:32.902513 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:32.902516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:32.903440 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:32.903450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:32.904333 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:32.904339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:32.904544 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:32.905256 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:31:32.905291 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:32.907703 (+ 2412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:32.910832 (+ 3129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:32.910840 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:32.910858 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:32.911220 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:32.911827 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:32.911832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:32.911840 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:32.912022 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:32.912733 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:32.912739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:32.913057 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 18:31:32.913248 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 18:31:32.913567 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":119,"threads_started":1}
I20260808 18:31:32.916270 659 heartbeater.cc:347] Connected to a master server at 127.7.164.126:46497
I20260808 18:31:32.916607 659 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:32.917210 659 heartbeater.cc:511] Master 127.7.164.126:46497 requested a full tablet report, sending...
I20260808 18:31:32.918566 32643 ts_manager.cc:194] Registered new tserver with Master: aec8c3050caa4d2fb56d01084555f82d (127.7.164.66:45275)
I20260808 18:31:32.919639 32643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:40295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:31:32.934854 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:46497
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:33.054176 665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:33.054534 665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:33.054626 665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:33.058764 665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:33.058847 665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:33.058938 665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:31:33.064333 665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:46497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:33.065466 665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:33.066601 665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:33.076725 674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:33.076655 676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:33.076655 672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:33.076764 665 server_base.cc:1034] running on GCE node
I20260808 18:31:33.077392 665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:33.078079 665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:33.079282 665 hybrid_clock.cc:648] HybridClock initialized: now 1786213893079261 us; error 38 us; skew 500 ppm
Aug 08 18:31:33 dist-test-slave-jztv krb5kdc[32582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213893, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:33.082168 665 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:31:33.083168 665 webserver.cc:460] Webserver started at http://127.7.164.67:39563/ using document root <none> and password file <none>
I20260808 18:31:33.083716 665 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:33.083776 665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:33.083947 665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:33.086045 665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "fce238b81bfd4912ac3608c5dfb79d41"
format_stamp: "Formatted at 2026-08-08 18:31:33 on dist-test-slave-jztv"
server_key: "645b65c56809fb9a69bd0bddfb67a8dc"
server_key_iv: "d78dc1b17d9490e802ab099b891a4439"
server_key_version: "encryptionkey@0"
I20260808 18:31:33.086669 665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "fce238b81bfd4912ac3608c5dfb79d41"
format_stamp: "Formatted at 2026-08-08 18:31:33 on dist-test-slave-jztv"
server_key: "645b65c56809fb9a69bd0bddfb67a8dc"
server_key_iv: "d78dc1b17d9490e802ab099b891a4439"
server_key_version: "encryptionkey@0"
I20260808 18:31:33.090513 665 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:31:33.093209 686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:33.094610 665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:31:33.094717 665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "fce238b81bfd4912ac3608c5dfb79d41"
format_stamp: "Formatted at 2026-08-08 18:31:33 on dist-test-slave-jztv"
server_key: "645b65c56809fb9a69bd0bddfb67a8dc"
server_key_iv: "d78dc1b17d9490e802ab099b891a4439"
server_key_version: "encryptionkey@0"
I20260808 18:31:33.094808 665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:33.138049 665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:33.141635 665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:33.141810 665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:33.142493 665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:33.143376 665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:33.143429 665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:33.143472 665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:33.143518 665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:33.153674 665 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:36483
I20260808 18:31:33.153700 847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:36483 every 8 connection(s)
I20260808 18:31:33.154706 665 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:31:33.164104 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 665
I20260808 18:31:33.164300 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:31:33.164618 7825 external_mini_cluster.cc:1428] Setting key 4e714fef4223d1b0439721f7d14d82f6
Aug 08 18:31:33 dist-test-slave-jztv krb5kdc[32582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213893, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:31:33.169334 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.156631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:36341 (local address 127.7.164.126:46497)
0808 18:31:33.156782 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:33.156785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:33.157654 (+ 869us) server_negotiation.cc:407] Connection header received
0808 18:31:33.159240 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:33.159244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:33.159306 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:33.159430 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:33.160744 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.161292 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.162545 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.162766 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.162878 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.165486 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:33.165490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:33.165492 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:33.165519 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:33.167204 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:33.167749 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:33.167753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:33.167755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:33.167840 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:33.168197 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:33.168202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:33.168203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:33.168398 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:33.168519 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:33.169037 (+ 518us) server_negotiation.cc:299] Negotiation successful
0808 18:31:33.169166 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:31:33.170140 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.157034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:46497 (local address 127.7.164.67:36341)
0808 18:31:33.157498 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:33.157541 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:33.158980 (+ 1439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:33.159618 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:33.159625 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:33.159993 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:33.160575 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:33.160590 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.161439 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.161443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:33.162420 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:33.162431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.162923 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.162927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:33.163100 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.163739 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:31:33.163757 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:33.165301 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:33.167336 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:33.167343 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:33.167360 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:33.167638 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:33.167939 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:33.167943 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:33.167945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:33.168096 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:33.168523 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:33.168531 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:33.168786 (+ 255us) client_negotiation.cc:769] Sending connection context
0808 18:31:33.168984 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 18:31:33.169314 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":109,"threads_started":1}
I20260808 18:31:33.171293 848 heartbeater.cc:347] Connected to a master server at 127.7.164.126:46497
I20260808 18:31:33.171566 848 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:33.172086 848 heartbeater.cc:511] Master 127.7.164.126:46497 requested a full tablet report, sending...
I20260808 18:31:33.173151 32643 ts_manager.cc:194] Registered new tserver with Master: fce238b81bfd4912ac3608c5dfb79d41 (127.7.164.67:36483)
I20260808 18:31:33.173743 32643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:36341
I20260808 18:31:33.178874 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:31:33.188228 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.180954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60160 (local address 127.7.164.126:46497)
0808 18:31:33.181082 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:33.181085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:33.181268 (+ 183us) server_negotiation.cc:407] Connection header received
0808 18:31:33.181368 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:33.181370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:33.181406 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:33.181474 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:33.182181 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.182746 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.183642 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.183854 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.183956 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.185338 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:33.185341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:33.185343 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:33.185366 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:33.186719 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:33.187128 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:33.187132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:33.187133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:33.187234 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:33.187488 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:33.187490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:33.187491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:33.187700 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:33.187805 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:33.188016 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 18:31:33.188111 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:31:33.191265 32643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:31:33.194398 32643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:31:33.209998 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.204066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:36483 (local address 127.0.0.1:43274)
0808 18:31:33.205034 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:33.205048 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:33.205139 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:33.205865 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:33.205868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:33.205890 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:33.205990 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:33.205996 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.207347 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.207351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:33.208293 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:33.208304 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.209381 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.209385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:33.209507 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.209551 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 18:31:33.209677 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 18:31:33.209808 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"thread_start_us":100,"threads_started":1}
I20260808 18:31:33.210016 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.204610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43274 (local address 127.7.164.67:36483)
0808 18:31:33.205391 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:33.205402 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:33.205430 (+ 28us) server_negotiation.cc:407] Connection header received
0808 18:31:33.205521 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:33.205528 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:33.205702 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:33.205855 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:33.206119 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.207202 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.208428 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.209183 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.209333 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.209482 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:33.209730 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 18:31:33.209846 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":163,"threads_started":1}
I20260808 18:31:33.210839 872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.203670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:38227 (local address 127.0.0.1:48566)
0808 18:31:33.204253 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:33.204268 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:33.204393 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:33.205275 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:33.205278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:33.205296 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:33.205422 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:33.205429 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.206893 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.206897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:33.208904 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:33.208917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.210389 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.210395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:33.210512 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.210537 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:31:33.210614 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 18:31:33.210687 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":366,"thread_start_us":92,"threads_started":1}
I20260808 18:31:33.211284 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.204232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48566 (local address 127.7.164.65:38227)
0808 18:31:33.204790 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:33.204799 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:33.204822 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:31:33.204898 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:33.204905 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:33.205120 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:33.205266 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:33.205549 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.206764 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.209058 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.209891 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.209991 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.210156 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:33.211036 (+ 880us) server_negotiation.cc:299] Negotiation successful
0808 18:31:33.211140 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":90,"threads_started":1}
I20260808 18:31:33.211429 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.203334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:45275 (local address 127.0.0.1:38364)
0808 18:31:33.203694 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:33.203740 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:33.203862 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:33.205083 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:33.205088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:33.205111 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:33.205275 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:33.205283 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.206718 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.206724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:33.208904 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:33.208917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.210676 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.210682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:33.210829 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.210884 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 18:31:33.211158 (+ 274us) client_negotiation.cc:240] Negotiation successful
0808 18:31:33.211243 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":376,"spinlock_wait_cycles":1792,"thread_start_us":107,"threads_started":1}
I20260808 18:31:33.211971 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.203489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.7.164.66:45275)
0808 18:31:33.204274 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:33.204291 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:33.204320 (+ 29us) server_negotiation.cc:407] Connection header received
0808 18:31:33.204409 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:33.204419 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:33.204675 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:33.204868 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:33.205429 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.206574 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.209103 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.210048 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.210247 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.210600 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:33.211662 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0808 18:31:33.211809 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":154,"threads_started":1}
I20260808 18:31:33.213954 755 tablet_service.cc:1467] Processing CreateTablet for tablet 2611f0d41d8244d0ba56a3c6003ec273 (DEFAULT_TABLE table=test-table [id=1cb7a3a81f004a13b0d704cc3f106f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:33.214002 356 tablet_service.cc:1467] Processing CreateTablet for tablet 2611f0d41d8244d0ba56a3c6003ec273 (DEFAULT_TABLE table=test-table [id=1cb7a3a81f004a13b0d704cc3f106f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:33.214617 565 tablet_service.cc:1467] Processing CreateTablet for tablet 2611f0d41d8244d0ba56a3c6003ec273 (DEFAULT_TABLE table=test-table [id=1cb7a3a81f004a13b0d704cc3f106f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:33.215142 755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2611f0d41d8244d0ba56a3c6003ec273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:33.215154 356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2611f0d41d8244d0ba56a3c6003ec273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:33.215535 565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2611f0d41d8244d0ba56a3c6003ec273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:33.221820 878 tablet_bootstrap.cc:492] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41: Bootstrap starting.
I20260808 18:31:33.223047 879 tablet_bootstrap.cc:492] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da: Bootstrap starting.
I20260808 18:31:33.224442 878 tablet_bootstrap.cc:654] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:33.224541 881 tablet_bootstrap.cc:492] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d: Bootstrap starting.
I20260808 18:31:33.225235 879 tablet_bootstrap.cc:654] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:33.225771 878 log.cc:824] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:33.226079 879 log.cc:824] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:33.227329 881 tablet_bootstrap.cc:654] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:33.228197 878 tablet_bootstrap.cc:492] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41: No bootstrap required, opened a new log
I20260808 18:31:33.228164 881 log.cc:824] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:33.228400 878 ts_tablet_manager.cc:1397] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 18:31:33.229355 879 tablet_bootstrap.cc:492] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da: No bootstrap required, opened a new log
I20260808 18:31:33.229598 879 ts_tablet_manager.cc:1397] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 18:31:33.230126 881 tablet_bootstrap.cc:492] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d: No bootstrap required, opened a new log
I20260808 18:31:33.230335 881 ts_tablet_manager.cc:1397] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:31:33.232592 878 raft_consensus.cc:348] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec8c3050caa4d2fb56d01084555f82d" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45275 } } peers { permanent_uuid: "fce238b81bfd4912ac3608c5dfb79d41" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36483 } } peers { permanent_uuid: "8356457ff32944c0bed9201ef4d968da" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38227 } }
I20260808 18:31:33.232894 878 raft_consensus.cc:374] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:33.232949 878 raft_consensus.cc:729] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fce238b81bfd4912ac3608c5dfb79d41, State: Initialized, Role: FOLLOWER
I20260808 18:31:33.233457 878 consensus_queue.cc:260] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec8c3050caa4d2fb56d01084555f82d" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45275 } } peers { permanent_uuid: "fce238b81bfd4912ac3608c5dfb79d41" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36483 } } peers { permanent_uuid: "8356457ff32944c0bed9201ef4d968da" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38227 } }
I20260808 18:31:33.233652 879 raft_consensus.cc:348] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec8c3050caa4d2fb56d01084555f82d" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45275 } } peers { permanent_uuid: "fce238b81bfd4912ac3608c5dfb79d41" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36483 } } peers { permanent_uuid: "8356457ff32944c0bed9201ef4d968da" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38227 } }
I20260808 18:31:33.233968 879 raft_consensus.cc:374] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:33.234040 879 raft_consensus.cc:729] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8356457ff32944c0bed9201ef4d968da, State: Initialized, Role: FOLLOWER
I20260808 18:31:33.234138 881 raft_consensus.cc:348] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec8c3050caa4d2fb56d01084555f82d" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45275 } } peers { permanent_uuid: "fce238b81bfd4912ac3608c5dfb79d41" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36483 } } peers { permanent_uuid: "8356457ff32944c0bed9201ef4d968da" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38227 } }
I20260808 18:31:33.234407 881 raft_consensus.cc:374] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:33.234458 881 raft_consensus.cc:729] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aec8c3050caa4d2fb56d01084555f82d, State: Initialized, Role: FOLLOWER
I20260808 18:31:33.234612 878 ts_tablet_manager.cc:1428] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260808 18:31:33.234550 879 consensus_queue.cc:260] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec8c3050caa4d2fb56d01084555f82d" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45275 } } peers { permanent_uuid: "fce238b81bfd4912ac3608c5dfb79d41" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36483 } } peers { permanent_uuid: "8356457ff32944c0bed9201ef4d968da" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38227 } }
I20260808 18:31:33.234903 881 consensus_queue.cc:260] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec8c3050caa4d2fb56d01084555f82d" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45275 } } peers { permanent_uuid: "fce238b81bfd4912ac3608c5dfb79d41" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36483 } } peers { permanent_uuid: "8356457ff32944c0bed9201ef4d968da" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38227 } }
I20260808 18:31:33.234999 848 heartbeater.cc:503] Master 127.7.164.126:46497 was elected leader, sending a full tablet report...
I20260808 18:31:33.235751 659 heartbeater.cc:503] Master 127.7.164.126:46497 was elected leader, sending a full tablet report...
I20260808 18:31:33.235924 881 ts_tablet_manager.cc:1428] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:31:33.236596 459 heartbeater.cc:503] Master 127.7.164.126:46497 was elected leader, sending a full tablet report...
I20260808 18:31:33.236919 879 ts_tablet_manager.cc:1428] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260808 18:31:33.395346 661 tablet.cc:2369] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:31:33.405725 851 tablet.cc:2369] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:31:33.417097 460 tablet.cc:2369] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:31:33.420217 887 raft_consensus.cc:482] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:31:33.420455 887 raft_consensus.cc:504] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec8c3050caa4d2fb56d01084555f82d" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45275 } } peers { permanent_uuid: "fce238b81bfd4912ac3608c5dfb79d41" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36483 } } peers { permanent_uuid: "8356457ff32944c0bed9201ef4d968da" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38227 } }
I20260808 18:31:33.421687 887 leader_election.cc:288] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aec8c3050caa4d2fb56d01084555f82d (127.7.164.66:45275), 8356457ff32944c0bed9201ef4d968da (127.7.164.65:38227)
I20260808 18:31:33.425524 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.422319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:43387 (local address 127.7.164.66:45275)
0808 18:31:33.422455 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:33.422459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:33.422474 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:31:33.422534 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:33.422536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:33.422578 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:33.422680 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:33.422983 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.423489 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.424549 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.424949 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.425084 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.425141 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:33.425327 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 18:31:33.425398 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:31:33.425524 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.422035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:45275 (local address 127.7.164.67:43387)
0808 18:31:33.422179 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:33.422196 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:33.422337 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:33.422697 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:33.422700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:33.422720 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:33.422842 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:33.422848 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.423611 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.423615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:33.424400 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:33.424409 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.425079 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.425082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:33.425195 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.425213 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 18:31:33.425310 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 18:31:33.425386 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 18:31:33.425838 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.422143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:43867 (local address 127.7.164.65:38227)
0808 18:31:33.422297 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:33.422302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:33.422636 (+ 334us) server_negotiation.cc:407] Connection header received
0808 18:31:33.422786 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:33.422788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:33.422831 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:33.422920 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:33.423202 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.423698 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.424771 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.425200 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.425348 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.425404 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:33.425613 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 18:31:33.425722 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:31:33.426131 894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.422035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:38227 (local address 127.7.164.67:43867)
0808 18:31:33.422528 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:33.422539 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:33.422663 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:33.422936 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:33.422939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:33.422956 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:33.423087 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:33.423092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.423815 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.423821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:33.424636 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:33.424644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.425344 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.425348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:33.425468 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.425497 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 18:31:33.425888 (+ 391us) client_negotiation.cc:240] Negotiation successful
0808 18:31:33.425967 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":49,"thread_start_us":164,"threads_started":1}
I20260808 18:31:33.426354 592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2611f0d41d8244d0ba56a3c6003ec273" candidate_uuid: "fce238b81bfd4912ac3608c5dfb79d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aec8c3050caa4d2fb56d01084555f82d" is_pre_election: true
I20260808 18:31:33.426709 592 raft_consensus.cc:2454] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fce238b81bfd4912ac3608c5dfb79d41 in term 0.
I20260808 18:31:33.426757 387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2611f0d41d8244d0ba56a3c6003ec273" candidate_uuid: "fce238b81bfd4912ac3608c5dfb79d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8356457ff32944c0bed9201ef4d968da" is_pre_election: true
I20260808 18:31:33.427160 387 raft_consensus.cc:2454] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fce238b81bfd4912ac3608c5dfb79d41 in term 0.
I20260808 18:31:33.427331 689 leader_election.cc:302] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aec8c3050caa4d2fb56d01084555f82d, fce238b81bfd4912ac3608c5dfb79d41; no voters:
I20260808 18:31:33.427645 887 raft_consensus.cc:2781] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:31:33.427754 887 raft_consensus.cc:482] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:31:33.427815 887 raft_consensus.cc:3037] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:33.428886 887 raft_consensus.cc:504] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec8c3050caa4d2fb56d01084555f82d" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45275 } } peers { permanent_uuid: "fce238b81bfd4912ac3608c5dfb79d41" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36483 } } peers { permanent_uuid: "8356457ff32944c0bed9201ef4d968da" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38227 } }
I20260808 18:31:33.429293 887 leader_election.cc:288] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [CANDIDATE]: Term 1 election: Requested vote from peers aec8c3050caa4d2fb56d01084555f82d (127.7.164.66:45275), 8356457ff32944c0bed9201ef4d968da (127.7.164.65:38227)
I20260808 18:31:33.429728 592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2611f0d41d8244d0ba56a3c6003ec273" candidate_uuid: "fce238b81bfd4912ac3608c5dfb79d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aec8c3050caa4d2fb56d01084555f82d"
I20260808 18:31:33.429806 387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2611f0d41d8244d0ba56a3c6003ec273" candidate_uuid: "fce238b81bfd4912ac3608c5dfb79d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8356457ff32944c0bed9201ef4d968da"
I20260808 18:31:33.429884 592 raft_consensus.cc:3037] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:33.429986 387 raft_consensus.cc:3037] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:33.431048 592 raft_consensus.cc:2454] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fce238b81bfd4912ac3608c5dfb79d41 in term 1.
I20260808 18:31:33.431188 387 raft_consensus.cc:2454] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fce238b81bfd4912ac3608c5dfb79d41 in term 1.
I20260808 18:31:33.431411 689 leader_election.cc:302] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aec8c3050caa4d2fb56d01084555f82d, fce238b81bfd4912ac3608c5dfb79d41; no voters:
I20260808 18:31:33.431589 887 raft_consensus.cc:2781] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:33.431833 887 raft_consensus.cc:686] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [term 1 LEADER]: Becoming Leader. State: Replica: fce238b81bfd4912ac3608c5dfb79d41, State: Running, Role: LEADER
I20260808 18:31:33.432173 887 consensus_queue.cc:237] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec8c3050caa4d2fb56d01084555f82d" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45275 } } peers { permanent_uuid: "fce238b81bfd4912ac3608c5dfb79d41" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36483 } } peers { permanent_uuid: "8356457ff32944c0bed9201ef4d968da" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38227 } }
I20260808 18:31:33.435734 32643 catalog_manager.cc:5495] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fce238b81bfd4912ac3608c5dfb79d41 (127.7.164.67). New cstate: current_term: 1 leader_uuid: "fce238b81bfd4912ac3608c5dfb79d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec8c3050caa4d2fb56d01084555f82d" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fce238b81bfd4912ac3608c5dfb79d41" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8356457ff32944c0bed9201ef4d968da" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38227 } health_report { overall_health: UNKNOWN } } }
I20260808 18:31:33.455543 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.451813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.7.164.67:36483)
0808 18:31:33.451954 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:33.451958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:33.451970 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:31:33.452055 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:33.452058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:33.452112 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:33.452211 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:31:33.452471 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.453009 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.453927 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.454152 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.454350 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.454694 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:31:33.455159 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:31:33.455267 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:33.455331 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 18:31:33.455387 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:31:33.461807 592 raft_consensus.cc:1261] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d [term 1 FOLLOWER]: Refusing update from remote peer fce238b81bfd4912ac3608c5dfb79d41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:31:33.461792 387 raft_consensus.cc:1261] T 2611f0d41d8244d0ba56a3c6003ec273 P 8356457ff32944c0bed9201ef4d968da [term 1 FOLLOWER]: Refusing update from remote peer fce238b81bfd4912ac3608c5dfb79d41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:31:33.462610 895 consensus_queue.cc:1035] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8356457ff32944c0bed9201ef4d968da" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:31:33.462811 887 consensus_queue.cc:1035] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aec8c3050caa4d2fb56d01084555f82d" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:31:33.470197 899 mvcc.cc:205] Tried to move back new op lower bound from 7316332107611598848 to 7316332107500806144. Current Snapshot: MvccSnapshot[applied={T|T < 7316332107611598848}]
I20260808 18:31:33.474792 901 mvcc.cc:205] Tried to move back new op lower bound from 7316332107611598848 to 7316332107500806144. Current Snapshot: MvccSnapshot[applied={T|T < 7316332107611598848}]
I20260808 18:31:33.480311 32643 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:31:33.480515 32643 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:31:33.483206 32643 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1d772afd1274da1b4b03a88373ee14b: Sending DeleteTablet for 3 replicas of tablet 2611f0d41d8244d0ba56a3c6003ec273
I20260808 18:31:33.484143 755 tablet_service.cc:1514] Processing DeleteTablet for tablet 2611f0d41d8244d0ba56a3c6003ec273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:33 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:43274
I20260808 18:31:33.484148 356 tablet_service.cc:1514] Processing DeleteTablet for tablet 2611f0d41d8244d0ba56a3c6003ec273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:33 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:48566
I20260808 18:31:33.484181 565 tablet_service.cc:1514] Processing DeleteTablet for tablet 2611f0d41d8244d0ba56a3c6003ec273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:33 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:38364
I20260808 18:31:33.484671 917 tablet_replica.cc:330] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d: stopping tablet replica
I20260808 18:31:33.485031 919 tablet_replica.cc:330] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41: stopping tablet replica
I20260808 18:31:33.485066 917 raft_consensus.cc:2229] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:31:33.485328 919 raft_consensus.cc:2229] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:31:33.485421 917 raft_consensus.cc:2258] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:31:33.485483 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 32736
I20260808 18:31:33.485745 919 raft_consensus.cc:2258] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:31:33.489445 919 ts_tablet_manager.cc:1905] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:31:33.492331 919 ts_tablet_manager.cc:1918] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:31:33.492444 919 log.cc:1196] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/2611f0d41d8244d0ba56a3c6003ec273
I20260808 18:31:33.492693 917 ts_tablet_manager.cc:1905] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:31:33.492784 919 ts_tablet_manager.cc:1939] T 2611f0d41d8244d0ba56a3c6003ec273 P fce238b81bfd4912ac3608c5dfb79d41: Deleting consensus metadata
I20260808 18:31:33.494025 32625 catalog_manager.cc:4841] TS fce238b81bfd4912ac3608c5dfb79d41 (127.7.164.67:36483): tablet 2611f0d41d8244d0ba56a3c6003ec273 (table test-table [id=1cb7a3a81f004a13b0d704cc3f106f09]) successfully deleted
I20260808 18:31:33.496775 917 ts_tablet_manager.cc:1918] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:31:33.496892 917 log.cc:1196] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/2611f0d41d8244d0ba56a3c6003ec273
I20260808 18:31:33.497242 917 ts_tablet_manager.cc:1939] T 2611f0d41d8244d0ba56a3c6003ec273 P aec8c3050caa4d2fb56d01084555f82d: Deleting consensus metadata
I20260808 18:31:33.498656 32625 catalog_manager.cc:4841] TS aec8c3050caa4d2fb56d01084555f82d (127.7.164.66:45275): tablet 2611f0d41d8244d0ba56a3c6003ec273 (table test-table [id=1cb7a3a81f004a13b0d704cc3f106f09]) successfully deleted
W20260808 18:31:33.500960 32627 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:38227 (error 108)
I20260808 18:31:33.501933 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 466
I20260808 18:31:33.502110 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.501771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:38227 (local address 127.0.0.1:48580)
0808 18:31:33.501956 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:33.502027 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260808 18:31:33.505259 32627 catalog_manager.cc:4584] TS 8356457ff32944c0bed9201ef4d968da (127.7.164.65:38227): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2611f0d41d8244d0ba56a3c6003ec273: Network error: Client connection negotiation failed: client connection to 127.7.164.65:38227: connect: Connection refused (error 111)
I20260808 18:31:33.512001 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 665
I20260808 18:31:33.526652 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 32603
2026-08-08T18:31:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2712 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T18:31:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:33Z Disabled control of system clock
I20260808 18:31:33.566555 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:44575
--webserver_interface=127.7.164.126
--webserver_port=0
--builtin_ntp_servers=127.7.164.84:45041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:44575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:31:33.679483 926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:33.679745 926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:33.679780 926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:33.683666 926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:33.683725 926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:33.683740 926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:33.683756 926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:33.683771 926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:33.688464 926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:45041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:44575
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:44575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:33.689522 926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:33.690837 926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:33.699126 936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:33.699177 934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:33.699205 931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:33.699540 926 server_base.cc:1034] running on GCE node
I20260808 18:31:33.700160 926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:33.701298 926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:33.702484 926 hybrid_clock.cc:648] HybridClock initialized: now 1786213893702468 us; error 37 us; skew 500 ppm
I20260808 18:31:33.704371 926 webserver.cc:460] Webserver started at http://127.7.164.126:32831/ using document root <none> and password file <none>
I20260808 18:31:33.704955 926 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:33.705061 926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:33.705304 926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:33.707444 926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "b009f8dea86e4fa69ad495183f893fe6"
format_stamp: "Formatted at 2026-08-08 18:31:33 on dist-test-slave-jztv"
server_key: "2c0b47f3c4a05d47339d95c392cf359a"
server_key_iv: "ba0e9821fdae482d02010d4cc0d66d48"
server_key_version: "encryptionkey@0"
I20260808 18:31:33.708035 926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "b009f8dea86e4fa69ad495183f893fe6"
format_stamp: "Formatted at 2026-08-08 18:31:33 on dist-test-slave-jztv"
server_key: "2c0b47f3c4a05d47339d95c392cf359a"
server_key_iv: "ba0e9821fdae482d02010d4cc0d66d48"
server_key_version: "encryptionkey@0"
I20260808 18:31:33.711665 926 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:31:33.714185 945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:33.715569 926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:31:33.715703 926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "b009f8dea86e4fa69ad495183f893fe6"
format_stamp: "Formatted at 2026-08-08 18:31:33 on dist-test-slave-jztv"
server_key: "2c0b47f3c4a05d47339d95c392cf359a"
server_key_iv: "ba0e9821fdae482d02010d4cc0d66d48"
server_key_version: "encryptionkey@0"
I20260808 18:31:33.715811 926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:33.733527 926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:33.734263 926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:33.734503 926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:33.741382 926 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:44575
I20260808 18:31:33.741393 1015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:44575 every 8 connection(s)
I20260808 18:31:33.742404 926 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:33.745579 1018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:33.751287 1018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6: Bootstrap starting.
I20260808 18:31:33.752454 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 926
I20260808 18:31:33.752602 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:33.752885 7825 external_mini_cluster.cc:1428] Setting key 06216dd9ee8a776d19b7bfe9b8e51fb0
I20260808 18:31:33.754081 1018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:33.754935 1018 log.cc:824] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:33.756973 1018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6: No bootstrap required, opened a new log
I20260808 18:31:33.760601 1018 raft_consensus.cc:348] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b009f8dea86e4fa69ad495183f893fe6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44575 } }
I20260808 18:31:33.760869 1018 raft_consensus.cc:374] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:33.760931 1018 raft_consensus.cc:729] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b009f8dea86e4fa69ad495183f893fe6, State: Initialized, Role: FOLLOWER
I20260808 18:31:33.761469 1018 consensus_queue.cc:260] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b009f8dea86e4fa69ad495183f893fe6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44575 } }
I20260808 18:31:33.761641 1018 raft_consensus.cc:388] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:33.761737 1018 raft_consensus.cc:482] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:33.761859 1018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:33.762459 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:33.754027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34254 (local address 127.7.164.126:44575)
0808 18:31:33.754720 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:33.754734 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:33.754768 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:31:33.755480 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:33.755500 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:33.755831 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:33.756197 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:33.756637 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.757684 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.758662 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:33.759182 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:33.759372 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:33.760261 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:33.760266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:33.760279 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:33.760674 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:33.760729 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:33.760749 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:33.760859 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:33.761062 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 18:31:33.761342 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":181,"threads_started":1}
I20260808 18:31:33.763119 1018 raft_consensus.cc:504] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b009f8dea86e4fa69ad495183f893fe6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44575 } }
I20260808 18:31:33.763480 1018 leader_election.cc:302] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b009f8dea86e4fa69ad495183f893fe6; no voters:
I20260808 18:31:33.763844 1018 leader_election.cc:288] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:33.763976 1023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:33.764200 1023 raft_consensus.cc:686] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [term 1 LEADER]: Becoming Leader. State: Replica: b009f8dea86e4fa69ad495183f893fe6, State: Running, Role: LEADER
I20260808 18:31:33.764534 1023 consensus_queue.cc:237] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b009f8dea86e4fa69ad495183f893fe6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44575 } }
I20260808 18:31:33.765245 1018 sys_catalog.cc:564] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:33.765810 1023 sys_catalog.cc:455] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b009f8dea86e4fa69ad495183f893fe6. Latest consensus state: current_term: 1 leader_uuid: "b009f8dea86e4fa69ad495183f893fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b009f8dea86e4fa69ad495183f893fe6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44575 } } }
I20260808 18:31:33.765941 1023 sys_catalog.cc:458] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:33.766359 1033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:33.766484 1024 sys_catalog.cc:455] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b009f8dea86e4fa69ad495183f893fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b009f8dea86e4fa69ad495183f893fe6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 44575 } } }
I20260808 18:31:33.766680 1024 sys_catalog.cc:458] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:33.769372 1033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:33.775242 1033 catalog_manager.cc:1347] Generated new cluster ID: d509746444d94594b9edf5a4b2e551b8
I20260808 18:31:33.775326 1033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:33.803151 1033 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:33.804171 1033 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:33.811841 1033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6: Generated new TSK 0
I20260808 18:31:33.812775 1033 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:31:33.828106 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:44575
--builtin_ntp_servers=127.7.164.84:45041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:31:33.955193 1054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:33.955547 1054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:33.955615 1054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:33.959865 1054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:33.959942 1054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:33.960001 1054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:31:33.964927 1054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:45041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:44575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:33.966287 1054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:33.967837 1054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:33.979203 1063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:33.979187 1061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:33.979209 1060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:33.979465 1054 server_base.cc:1034] running on GCE node
I20260808 18:31:33.980068 1054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:33.980949 1054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:33.982179 1054 hybrid_clock.cc:648] HybridClock initialized: now 1786213893982137 us; error 53 us; skew 500 ppm
I20260808 18:31:33.984455 1054 webserver.cc:460] Webserver started at http://127.7.164.65:36435/ using document root <none> and password file <none>
I20260808 18:31:33.985121 1054 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:33.985226 1054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:33.985487 1054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:33.988148 1054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "3fc1b8efc2144d4daef20313b59d0653"
format_stamp: "Formatted at 2026-08-08 18:31:33 on dist-test-slave-jztv"
server_key: "aa9b56f9034e9bff3a055b8ee157bc01"
server_key_iv: "c72af1c342dffed4e77be9c828a35490"
server_key_version: "encryptionkey@0"
I20260808 18:31:33.988758 1054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "3fc1b8efc2144d4daef20313b59d0653"
format_stamp: "Formatted at 2026-08-08 18:31:33 on dist-test-slave-jztv"
server_key: "aa9b56f9034e9bff3a055b8ee157bc01"
server_key_iv: "c72af1c342dffed4e77be9c828a35490"
server_key_version: "encryptionkey@0"
I20260808 18:31:33.992741 1054 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:31:33.995564 1071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:33.996857 1054 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 18:31:33.997017 1054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "3fc1b8efc2144d4daef20313b59d0653"
format_stamp: "Formatted at 2026-08-08 18:31:33 on dist-test-slave-jztv"
server_key: "aa9b56f9034e9bff3a055b8ee157bc01"
server_key_iv: "c72af1c342dffed4e77be9c828a35490"
server_key_version: "encryptionkey@0"
I20260808 18:31:33.997248 1054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:34.011737 1054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:34.012457 1054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:34.012665 1054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:34.013285 1054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:34.014336 1054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:34.014406 1054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:34.014472 1054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:34.014523 1054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:34.024842 1054 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:41793
I20260808 18:31:34.024880 1233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:41793 every 8 connection(s)
I20260808 18:31:34.025841 1054 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:31:34.026954 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 1054
I20260808 18:31:34.027101 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:31:34.027396 7825 external_mini_cluster.cc:1428] Setting key 80b17cd32964b1d5102f71a4cb7d962b
I20260808 18:31:34.029615 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:44575
--builtin_ntp_servers=127.7.164.84:45041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:31:34.035990 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.027909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:47037 (local address 127.7.164.126:44575)
0808 18:31:34.028061 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:34.028065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:34.029388 (+ 1323us) server_negotiation.cc:407] Connection header received
0808 18:31:34.031030 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:34.031034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:34.031095 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:34.031177 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:34.032125 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.032684 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.033876 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.034131 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.034274 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.035445 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:34.035449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:34.035452 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:34.035466 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:34.035496 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:34.035511 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:34.035597 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:34.035766 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 18:31:34.035838 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:31:34.036893 1236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.028431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:44575 (local address 127.7.164.65:47037)
0808 18:31:34.029235 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:34.029277 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:34.030800 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:34.031359 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:34.031366 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:34.031454 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:34.031959 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.031972 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.032829 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.032837 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.033734 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.033746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.034332 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.034337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.034510 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.035164 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:31:34.035166 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:34.035208 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:31:34.035276 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:34.035599 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:34.035613 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:34.035622 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 18:31:34.035764 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 18:31:34.036081 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":148,"threads_started":1}
I20260808 18:31:34.038291 1234 heartbeater.cc:347] Connected to a master server at 127.7.164.126:44575
I20260808 18:31:34.038614 1234 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:34.039355 1234 heartbeater.cc:511] Master 127.7.164.126:44575 requested a full tablet report, sending...
I20260808 18:31:34.041258 974 ts_manager.cc:194] Registered new tserver with Master: 3fc1b8efc2144d4daef20313b59d0653 (127.7.164.65:41793)
I20260808 18:31:34.042896 974 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.164.65:47037
W20260808 18:31:34.145072 1238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:34.145339 1238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:34.145380 1238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:34.149310 1238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:34.149366 1238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:34.149420 1238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:31:34.154408 1238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:45041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:44575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:34.155516 1238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:34.156894 1238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:34.166805 1250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:34.166848 1248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:34.166824 1247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:34.167178 1238 server_base.cc:1034] running on GCE node
I20260808 18:31:34.167667 1238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:34.168394 1238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:34.169595 1238 hybrid_clock.cc:648] HybridClock initialized: now 1786213894169572 us; error 41 us; skew 500 ppm
I20260808 18:31:34.171598 1238 webserver.cc:460] Webserver started at http://127.7.164.66:45293/ using document root <none> and password file <none>
I20260808 18:31:34.172225 1238 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:34.172322 1238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:34.172585 1238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:34.174818 1238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "0aa7a47f1ac84682bdea1e7704e09ddb"
format_stamp: "Formatted at 2026-08-08 18:31:34 on dist-test-slave-jztv"
server_key: "209603845cff71f27b5666585dc023c7"
server_key_iv: "ab2bbec19acd9835c7c6d53e5725bf2d"
server_key_version: "encryptionkey@0"
I20260808 18:31:34.175416 1238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "0aa7a47f1ac84682bdea1e7704e09ddb"
format_stamp: "Formatted at 2026-08-08 18:31:34 on dist-test-slave-jztv"
server_key: "209603845cff71f27b5666585dc023c7"
server_key_iv: "ab2bbec19acd9835c7c6d53e5725bf2d"
server_key_version: "encryptionkey@0"
I20260808 18:31:34.179167 1238 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:31:34.181653 1258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:34.182878 1238 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:31:34.183019 1238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "0aa7a47f1ac84682bdea1e7704e09ddb"
format_stamp: "Formatted at 2026-08-08 18:31:34 on dist-test-slave-jztv"
server_key: "209603845cff71f27b5666585dc023c7"
server_key_iv: "ab2bbec19acd9835c7c6d53e5725bf2d"
server_key_version: "encryptionkey@0"
I20260808 18:31:34.183136 1238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:34.208690 1238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:34.209411 1238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:34.209635 1238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:34.210312 1238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:34.211306 1238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:34.211354 1238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:34.211395 1238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:34.211410 1238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:34.220992 1238 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:41377
I20260808 18:31:34.221027 1410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:41377 every 8 connection(s)
I20260808 18:31:34.221982 1238 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:31:34.227288 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 1238
I20260808 18:31:34.227439 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:31:34.227751 7825 external_mini_cluster.cc:1428] Setting key 0abc29ae76d55bd8517c4c7277ea09ed
I20260808 18:31:34.229727 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:44575
--builtin_ntp_servers=127.7.164.84:45041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:31:34.231875 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.224054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:55561 (local address 127.7.164.126:44575)
0808 18:31:34.224201 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:34.224205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:34.225273 (+ 1068us) server_negotiation.cc:407] Connection header received
0808 18:31:34.226763 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:34.226766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:34.226828 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:34.226947 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:34.228008 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.228560 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.229714 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.230030 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.230144 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.231362 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:34.231366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:34.231369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:34.231382 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:34.231405 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:34.231419 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:34.231498 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:34.231673 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 18:31:34.231735 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:31:34.232767 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.224413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:44575 (local address 127.7.164.66:55561)
0808 18:31:34.225095 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:34.225148 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:34.226546 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:34.227125 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:34.227132 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:34.227217 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:34.227856 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.227873 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.228695 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.228698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.229580 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.229590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.230189 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.230193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.230404 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.231091 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:31:34.231092 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:34.231133 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:31:34.231198 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:34.231513 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:34.231524 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:34.231537 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 18:31:34.231668 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 18:31:34.231955 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":133,"threads_started":1}
I20260808 18:31:34.234025 1411 heartbeater.cc:347] Connected to a master server at 127.7.164.126:44575
I20260808 18:31:34.234300 1411 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:34.234992 1411 heartbeater.cc:511] Master 127.7.164.126:44575 requested a full tablet report, sending...
I20260808 18:31:34.236207 974 ts_manager.cc:194] Registered new tserver with Master: 0aa7a47f1ac84682bdea1e7704e09ddb (127.7.164.66:41377)
I20260808 18:31:34.236806 974 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.164.66:55561
W20260808 18:31:34.350621 1414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:34.350881 1414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:34.350919 1414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:34.354894 1414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:34.354951 1414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:34.355006 1414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:31:34.359679 1414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:45041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:44575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:34.360675 1414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:34.361987 1414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:34.372081 1421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:34.372081 1428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:34.372076 1422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:34.372140 1414 server_base.cc:1034] running on GCE node
I20260808 18:31:34.372728 1414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:34.373381 1414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:34.374572 1414 hybrid_clock.cc:648] HybridClock initialized: now 1786213894374556 us; error 27 us; skew 500 ppm
I20260808 18:31:34.379320 1414 webserver.cc:460] Webserver started at http://127.7.164.67:39767/ using document root <none> and password file <none>
I20260808 18:31:34.379948 1414 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:34.380059 1414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:34.380301 1414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:34.382488 1414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "aff62ecc07e24ca4b66d5d815683bb46"
format_stamp: "Formatted at 2026-08-08 18:31:34 on dist-test-slave-jztv"
server_key: "1e75f25c8654e8e00c6a8758a1eb7b45"
server_key_iv: "3a37fc0c05c35ca1cdf6cc8e32f2e57b"
server_key_version: "encryptionkey@0"
I20260808 18:31:34.383070 1414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "aff62ecc07e24ca4b66d5d815683bb46"
format_stamp: "Formatted at 2026-08-08 18:31:34 on dist-test-slave-jztv"
server_key: "1e75f25c8654e8e00c6a8758a1eb7b45"
server_key_iv: "3a37fc0c05c35ca1cdf6cc8e32f2e57b"
server_key_version: "encryptionkey@0"
I20260808 18:31:34.386929 1414 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 18:31:34.389645 1442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:34.390870 1414 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:31:34.391011 1414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "aff62ecc07e24ca4b66d5d815683bb46"
format_stamp: "Formatted at 2026-08-08 18:31:34 on dist-test-slave-jztv"
server_key: "1e75f25c8654e8e00c6a8758a1eb7b45"
server_key_iv: "3a37fc0c05c35ca1cdf6cc8e32f2e57b"
server_key_version: "encryptionkey@0"
I20260808 18:31:34.391129 1414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:34.410848 1414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:34.411581 1414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:34.411794 1414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:34.412417 1414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:34.413412 1414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:34.413486 1414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:34.413558 1414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:34.413612 1414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:34.423131 1414 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:41929
I20260808 18:31:34.423174 1590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:41929 every 8 connection(s)
I20260808 18:31:34.424096 1414 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:31:34.428232 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 1414
I20260808 18:31:34.428382 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:31:34.428674 7825 external_mini_cluster.cc:1428] Setting key 345fd876ac7ec2ca2640ad728bc1516f
I20260808 18:31:34.434042 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.426014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:57595 (local address 127.7.164.126:44575)
0808 18:31:34.426157 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:34.426161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:34.427264 (+ 1103us) server_negotiation.cc:407] Connection header received
0808 18:31:34.428809 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:34.428813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:34.428876 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:34.428988 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:34.430061 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.430829 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.432000 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.432247 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.432363 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.433524 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:34.433528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:34.433531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:34.433544 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:34.433572 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:34.433580 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:34.433660 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:34.433822 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 18:31:34.433890 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:31:34.434942 1593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.426445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:44575 (local address 127.7.164.67:57595)
0808 18:31:34.427107 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:34.427148 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:34.428596 (+ 1448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:34.429195 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:34.429205 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:34.429309 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:34.429913 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.429928 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.430981 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.430985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.431862 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.431872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.432405 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.432409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.432581 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.433225 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:31:34.433227 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:34.433268 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:31:34.433332 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:34.433662 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:34.433674 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:34.433682 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 18:31:34.433822 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 18:31:34.434108 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":122,"threads_started":1}
I20260808 18:31:34.436064 1591 heartbeater.cc:347] Connected to a master server at 127.7.164.126:44575
I20260808 18:31:34.436299 1591 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:34.436815 1591 heartbeater.cc:511] Master 127.7.164.126:44575 requested a full tablet report, sending...
I20260808 18:31:34.437902 974 ts_manager.cc:194] Registered new tserver with Master: aff62ecc07e24ca4b66d5d815683bb46 (127.7.164.67:41929)
I20260808 18:31:34.438548 974 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.164.67:57595
I20260808 18:31:34.442785 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:31:34.448355 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.444731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34264 (local address 127.7.164.126:44575)
0808 18:31:34.444874 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:34.444878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:34.445067 (+ 189us) server_negotiation.cc:407] Connection header received
0808 18:31:34.445143 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:34.445146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:34.445185 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:34.445250 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:34.445494 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.445990 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.446897 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.447110 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.447190 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.447935 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:34.447938 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:34.447941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:34.447952 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:34.447970 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:34.447978 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:34.448043 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:34.448120 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 18:31:34.448167 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:31:34.451090 974 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:31:34.454072 974 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:31:34.469015 1616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.463567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47694 (local address 127.7.164.66:41377)
0808 18:31:34.463998 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:34.464010 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:34.464032 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:31:34.464112 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:34.464117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:34.464312 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:34.464495 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:34.464684 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.465910 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.467806 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.468315 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.468443 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.468653 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:34.468752 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 18:31:34.468846 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":124,"threads_started":1}
I20260808 18:31:34.469019 1610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.462775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:41377 (local address 127.0.0.1:47694)
0808 18:31:34.463295 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:34.463312 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:34.463414 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:34.464466 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:34.464471 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:34.464492 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:34.464585 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.464591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.466021 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.466024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.467650 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.467661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.468443 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.468447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.468561 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.468601 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 18:31:34.468725 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 18:31:34.468846 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":180,"thread_start_us":121,"threads_started":1}
I20260808 18:31:34.469105 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.462727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:41929 (local address 127.0.0.1:48624)
0808 18:31:34.463163 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:34.463209 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:34.463340 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:34.464209 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:34.464216 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:34.464240 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:34.464392 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.464398 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.465886 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.465893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.467650 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.467661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.468670 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.468674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.468762 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.468786 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:31:34.468837 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 18:31:34.468921 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":159,"thread_start_us":83,"threads_started":1}
I20260808 18:31:34.469445 1618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.463851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38440 (local address 127.7.164.65:41793)
0808 18:31:34.464264 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:34.464276 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:34.464608 (+ 332us) server_negotiation.cc:407] Connection header received
0808 18:31:34.464695 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:34.464700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:34.464877 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:34.465041 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:34.465278 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.466338 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.467589 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.468360 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.468486 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.468682 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:34.469175 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 18:31:34.469275 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":100,"threads_started":1}
I20260808 18:31:34.469740 1613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.462883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48624 (local address 127.7.164.67:41929)
0808 18:31:34.463680 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:34.463693 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:34.463719 (+ 26us) server_negotiation.cc:407] Connection header received
0808 18:31:34.463800 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:34.463805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:34.464020 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:34.464586 (+ 566us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:34.464749 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.465770 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.467791 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.468532 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.469231 (+ 699us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.469405 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:34.469461 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 18:31:34.469555 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":133,"threads_started":1}
I20260808 18:31:34.469105 1614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.463430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:41793 (local address 127.0.0.1:38440)
0808 18:31:34.464411 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:34.464426 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:34.464511 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:34.465049 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:34.465052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:34.465072 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:34.465150 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.465155 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.466470 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.466473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.467460 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.467470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.468734 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.468737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.468831 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.468852 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:31:34.468910 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 18:31:34.468961 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"mutex_wait_us":4,"thread_start_us":99,"threads_started":1}
I20260808 18:31:34.472538 1505 tablet_service.cc:1467] Processing CreateTablet for tablet d8539e5f03724fe8946f970567edafd8 (DEFAULT_TABLE table=test-table [id=7b467f50a0164428b58838baf20a6505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:34.472690 1325 tablet_service.cc:1467] Processing CreateTablet for tablet d8539e5f03724fe8946f970567edafd8 (DEFAULT_TABLE table=test-table [id=7b467f50a0164428b58838baf20a6505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:34.473104 1136 tablet_service.cc:1467] Processing CreateTablet for tablet d8539e5f03724fe8946f970567edafd8 (DEFAULT_TABLE table=test-table [id=7b467f50a0164428b58838baf20a6505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:34.473632 1505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8539e5f03724fe8946f970567edafd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:34.473764 1325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8539e5f03724fe8946f970567edafd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:34.474049 1136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8539e5f03724fe8946f970567edafd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:34.480288 1619 tablet_bootstrap.cc:492] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46: Bootstrap starting.
I20260808 18:31:34.481671 1622 tablet_bootstrap.cc:492] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb: Bootstrap starting.
I20260808 18:31:34.482514 1620 tablet_bootstrap.cc:492] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653: Bootstrap starting.
I20260808 18:31:34.482700 1619 tablet_bootstrap.cc:654] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:34.483584 1619 log.cc:824] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:34.483784 1622 tablet_bootstrap.cc:654] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:34.484375 1620 tablet_bootstrap.cc:654] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:34.484650 1622 log.cc:824] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:34.485188 1620 log.cc:824] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:34.486209 1619 tablet_bootstrap.cc:492] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46: No bootstrap required, opened a new log
I20260808 18:31:34.486464 1619 ts_tablet_manager.cc:1397] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260808 18:31:34.486693 1622 tablet_bootstrap.cc:492] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb: No bootstrap required, opened a new log
I20260808 18:31:34.486899 1622 ts_tablet_manager.cc:1397] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:31:34.487181 1620 tablet_bootstrap.cc:492] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653: No bootstrap required, opened a new log
I20260808 18:31:34.487350 1620 ts_tablet_manager.cc:1397] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:31:34.490084 1622 raft_consensus.cc:348] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff62ecc07e24ca4b66d5d815683bb46" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41929 } } peers { permanent_uuid: "0aa7a47f1ac84682bdea1e7704e09ddb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41377 } } peers { permanent_uuid: "3fc1b8efc2144d4daef20313b59d0653" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41793 } }
I20260808 18:31:34.490394 1622 raft_consensus.cc:374] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:34.490470 1622 raft_consensus.cc:729] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aa7a47f1ac84682bdea1e7704e09ddb, State: Initialized, Role: FOLLOWER
I20260808 18:31:34.490968 1622 consensus_queue.cc:260] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff62ecc07e24ca4b66d5d815683bb46" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41929 } } peers { permanent_uuid: "0aa7a47f1ac84682bdea1e7704e09ddb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41377 } } peers { permanent_uuid: "3fc1b8efc2144d4daef20313b59d0653" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41793 } }
I20260808 18:31:34.490980 1619 raft_consensus.cc:348] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff62ecc07e24ca4b66d5d815683bb46" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41929 } } peers { permanent_uuid: "0aa7a47f1ac84682bdea1e7704e09ddb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41377 } } peers { permanent_uuid: "3fc1b8efc2144d4daef20313b59d0653" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41793 } }
I20260808 18:31:34.491214 1619 raft_consensus.cc:374] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:34.491262 1619 raft_consensus.cc:729] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aff62ecc07e24ca4b66d5d815683bb46, State: Initialized, Role: FOLLOWER
I20260808 18:31:34.491403 1620 raft_consensus.cc:348] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff62ecc07e24ca4b66d5d815683bb46" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41929 } } peers { permanent_uuid: "0aa7a47f1ac84682bdea1e7704e09ddb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41377 } } peers { permanent_uuid: "3fc1b8efc2144d4daef20313b59d0653" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41793 } }
I20260808 18:31:34.491626 1620 raft_consensus.cc:374] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:34.491669 1620 raft_consensus.cc:729] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc1b8efc2144d4daef20313b59d0653, State: Initialized, Role: FOLLOWER
I20260808 18:31:34.491665 1619 consensus_queue.cc:260] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff62ecc07e24ca4b66d5d815683bb46" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41929 } } peers { permanent_uuid: "0aa7a47f1ac84682bdea1e7704e09ddb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41377 } } peers { permanent_uuid: "3fc1b8efc2144d4daef20313b59d0653" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41793 } }
I20260808 18:31:34.491986 1622 ts_tablet_manager.cc:1428] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:31:34.492115 1620 consensus_queue.cc:260] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff62ecc07e24ca4b66d5d815683bb46" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41929 } } peers { permanent_uuid: "0aa7a47f1ac84682bdea1e7704e09ddb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41377 } } peers { permanent_uuid: "3fc1b8efc2144d4daef20313b59d0653" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41793 } }
I20260808 18:31:34.492352 1411 heartbeater.cc:503] Master 127.7.164.126:44575 was elected leader, sending a full tablet report...
I20260808 18:31:34.492599 1619 ts_tablet_manager.cc:1428] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260808 18:31:34.492945 1591 heartbeater.cc:503] Master 127.7.164.126:44575 was elected leader, sending a full tablet report...
I20260808 18:31:34.493034 1620 ts_tablet_manager.cc:1428] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:31:34.493369 1234 heartbeater.cc:503] Master 127.7.164.126:44575 was elected leader, sending a full tablet report...
I20260808 18:31:34.501261 1629 raft_consensus.cc:482] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:31:34.501421 1629 raft_consensus.cc:504] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff62ecc07e24ca4b66d5d815683bb46" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41929 } } peers { permanent_uuid: "0aa7a47f1ac84682bdea1e7704e09ddb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41377 } } peers { permanent_uuid: "3fc1b8efc2144d4daef20313b59d0653" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41793 } }
I20260808 18:31:34.502619 1629 leader_election.cc:288] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aff62ecc07e24ca4b66d5d815683bb46 (127.7.164.67:41929), 0aa7a47f1ac84682bdea1e7704e09ddb (127.7.164.66:41377)
I20260808 18:31:34.506222 1236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.502771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:41929 (local address 127.7.164.65:35957)
0808 18:31:34.502883 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:34.502899 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:34.502981 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:34.503231 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:34.503234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:34.503256 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:34.503378 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.503385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.504232 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.504235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.505115 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.505125 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.505790 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.505793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.505905 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.505931 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 18:31:34.506015 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 18:31:34.506071 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":26}
I20260808 18:31:34.506310 1613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.502832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:35957 (local address 127.7.164.67:41929)
0808 18:31:34.502959 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:34.502963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:34.502980 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:31:34.503081 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:34.503086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:34.503137 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:34.503205 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:34.503507 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.504124 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.505243 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.505675 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.505768 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.505805 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:34.506052 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 18:31:34.506134 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:31:34.506395 1616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.503208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:54759 (local address 127.7.164.66:41377)
0808 18:31:34.503316 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:34.503320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:34.503332 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:31:34.503375 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:34.503377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:34.503427 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:34.503500 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:34.503788 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.504300 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.505372 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.505801 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.505914 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.505955 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:34.506177 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 18:31:34.506285 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:31:34.506464 1630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.502838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:41377 (local address 127.7.164.65:54759)
0808 18:31:34.503059 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:34.503072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:34.503153 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:34.503531 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:34.503535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:34.503553 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:34.503678 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.503685 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.504415 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.504417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.505229 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:34.505239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.505932 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.505936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:34.506045 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.506071 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 18:31:34.506142 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 18:31:34.506204 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":25,"thread_start_us":73,"threads_started":1}
I20260808 18:31:34.506994 1530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8539e5f03724fe8946f970567edafd8" candidate_uuid: "3fc1b8efc2144d4daef20313b59d0653" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aff62ecc07e24ca4b66d5d815683bb46" is_pre_election: true
I20260808 18:31:34.507004 1355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8539e5f03724fe8946f970567edafd8" candidate_uuid: "3fc1b8efc2144d4daef20313b59d0653" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0aa7a47f1ac84682bdea1e7704e09ddb" is_pre_election: true
I20260808 18:31:34.507253 1355 raft_consensus.cc:2454] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc1b8efc2144d4daef20313b59d0653 in term 0.
I20260808 18:31:34.507256 1530 raft_consensus.cc:2454] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fc1b8efc2144d4daef20313b59d0653 in term 0.
I20260808 18:31:34.507764 1077 leader_election.cc:302] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0aa7a47f1ac84682bdea1e7704e09ddb, 3fc1b8efc2144d4daef20313b59d0653, aff62ecc07e24ca4b66d5d815683bb46; no voters:
I20260808 18:31:34.508030 1629 raft_consensus.cc:2781] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:31:34.508104 1629 raft_consensus.cc:482] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:31:34.508155 1629 raft_consensus.cc:3037] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:34.509121 1629 raft_consensus.cc:504] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff62ecc07e24ca4b66d5d815683bb46" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41929 } } peers { permanent_uuid: "0aa7a47f1ac84682bdea1e7704e09ddb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41377 } } peers { permanent_uuid: "3fc1b8efc2144d4daef20313b59d0653" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41793 } }
I20260808 18:31:34.509498 1629 leader_election.cc:288] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [CANDIDATE]: Term 1 election: Requested vote from peers aff62ecc07e24ca4b66d5d815683bb46 (127.7.164.67:41929), 0aa7a47f1ac84682bdea1e7704e09ddb (127.7.164.66:41377)
I20260808 18:31:34.509784 1530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8539e5f03724fe8946f970567edafd8" candidate_uuid: "3fc1b8efc2144d4daef20313b59d0653" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aff62ecc07e24ca4b66d5d815683bb46"
I20260808 18:31:34.509848 1355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8539e5f03724fe8946f970567edafd8" candidate_uuid: "3fc1b8efc2144d4daef20313b59d0653" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0aa7a47f1ac84682bdea1e7704e09ddb"
I20260808 18:31:34.509907 1530 raft_consensus.cc:3037] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:34.509990 1355 raft_consensus.cc:3037] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:34.511040 1530 raft_consensus.cc:2454] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc1b8efc2144d4daef20313b59d0653 in term 1.
I20260808 18:31:34.511119 1355 raft_consensus.cc:2454] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fc1b8efc2144d4daef20313b59d0653 in term 1.
I20260808 18:31:34.511392 1074 leader_election.cc:302] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fc1b8efc2144d4daef20313b59d0653, aff62ecc07e24ca4b66d5d815683bb46; no voters:
I20260808 18:31:34.511595 1629 raft_consensus.cc:2781] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:34.511826 1629 raft_consensus.cc:686] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [term 1 LEADER]: Becoming Leader. State: Replica: 3fc1b8efc2144d4daef20313b59d0653, State: Running, Role: LEADER
I20260808 18:31:34.512122 1629 consensus_queue.cc:237] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff62ecc07e24ca4b66d5d815683bb46" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41929 } } peers { permanent_uuid: "0aa7a47f1ac84682bdea1e7704e09ddb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41377 } } peers { permanent_uuid: "3fc1b8efc2144d4daef20313b59d0653" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41793 } }
I20260808 18:31:34.515021 966 catalog_manager.cc:5495] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fc1b8efc2144d4daef20313b59d0653 (127.7.164.65). New cstate: current_term: 1 leader_uuid: "3fc1b8efc2144d4daef20313b59d0653" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff62ecc07e24ca4b66d5d815683bb46" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0aa7a47f1ac84682bdea1e7704e09ddb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fc1b8efc2144d4daef20313b59d0653" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 41793 } health_report { overall_health: HEALTHY } } }
W20260808 18:31:34.527148 1235 tablet.cc:2369] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:31:34.530701 1618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.527383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38448 (local address 127.7.164.65:41793)
0808 18:31:34.527527 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:34.527531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:34.527545 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:31:34.527692 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:34.527694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:34.527750 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:34.527833 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:31:34.528072 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.528582 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.529376 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:34.529584 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:34.529657 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:34.529909 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:31:34.530380 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:31:34.530472 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:34.530538 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 18:31:34.530582 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:31:34.536391 1355 raft_consensus.cc:1261] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb [term 1 FOLLOWER]: Refusing update from remote peer 3fc1b8efc2144d4daef20313b59d0653: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:31:34.536391 1530 raft_consensus.cc:1261] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46 [term 1 FOLLOWER]: Refusing update from remote peer 3fc1b8efc2144d4daef20313b59d0653: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:31:34.537127 1631 consensus_queue.cc:1035] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aff62ecc07e24ca4b66d5d815683bb46" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:31:34.537314 1629 consensus_queue.cc:1035] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0aa7a47f1ac84682bdea1e7704e09ddb" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:31:34.545271 1640 mvcc.cc:205] Tried to move back new op lower bound from 7316332112013959168 to 7316332111924092928. Current Snapshot: MvccSnapshot[applied={T|T < 7316332112013959168}]
I20260808 18:31:34.553901 974 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:31:34.554066 974 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:31:34.556530 974 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b009f8dea86e4fa69ad495183f893fe6: Sending DeleteTablet for 3 replicas of tablet d8539e5f03724fe8946f970567edafd8
I20260808 18:31:34.557336 1505 tablet_service.cc:1514] Processing DeleteTablet for tablet d8539e5f03724fe8946f970567edafd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:34 UTC) from {username='slave'} at 127.0.0.1:48624
I20260808 18:31:34.557430 1136 tablet_service.cc:1514] Processing DeleteTablet for tablet d8539e5f03724fe8946f970567edafd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:34 UTC) from {username='slave'} at 127.0.0.1:38440
I20260808 18:31:34.557894 1651 tablet_replica.cc:330] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653: stopping tablet replica
I20260808 18:31:34.557899 1325 tablet_service.cc:1514] Processing DeleteTablet for tablet d8539e5f03724fe8946f970567edafd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:34 UTC) from {username='slave'} at 127.0.0.1:47694
I20260808 18:31:34.558360 1651 raft_consensus.cc:2229] T d8539e5f03724fe8946f970567edafd8 P 3fc1b8efc2144d4daef20313b59d0653 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:31:34.558385 1653 tablet_replica.cc:330] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb: stopping tablet replica
I20260808 18:31:34.558624 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 1054
I20260808 18:31:34.562093 1652 tablet_replica.cc:330] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46: stopping tablet replica
I20260808 18:31:34.562414 1652 raft_consensus.cc:2229] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:31:34.562742 1652 raft_consensus.cc:2258] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:31:34.565248 1652 ts_tablet_manager.cc:1905] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:31:34.569074 1653 raft_consensus.cc:2229] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:31:34.569442 1653 raft_consensus.cc:2258] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:31:34.569638 1652 ts_tablet_manager.cc:1918] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:31:34.569746 1652 log.cc:1196] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/d8539e5f03724fe8946f970567edafd8
I20260808 18:31:34.570139 1652 ts_tablet_manager.cc:1939] T d8539e5f03724fe8946f970567edafd8 P aff62ecc07e24ca4b66d5d815683bb46: Deleting consensus metadata
I20260808 18:31:34.571128 1653 ts_tablet_manager.cc:1905] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:31:34.571408 948 catalog_manager.cc:4841] TS aff62ecc07e24ca4b66d5d815683bb46 (127.7.164.67:41929): tablet d8539e5f03724fe8946f970567edafd8 (table test-table [id=7b467f50a0164428b58838baf20a6505]) successfully deleted
I20260808 18:31:34.573312 1653 ts_tablet_manager.cc:1918] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:31:34.573438 1653 log.cc:1196] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/d8539e5f03724fe8946f970567edafd8
I20260808 18:31:34.573735 1653 ts_tablet_manager.cc:1939] T d8539e5f03724fe8946f970567edafd8 P 0aa7a47f1ac84682bdea1e7704e09ddb: Deleting consensus metadata
I20260808 18:31:34.574796 950 catalog_manager.cc:4841] TS 0aa7a47f1ac84682bdea1e7704e09ddb (127.7.164.66:41377): tablet d8539e5f03724fe8946f970567edafd8 (table test-table [id=7b467f50a0164428b58838baf20a6505]) successfully deleted
W20260808 18:31:34.575398 950 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:41793 (error 108)
I20260808 18:31:34.575688 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 1238
I20260808 18:31:34.583225 1614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:34.582847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:41793 (local address 127.0.0.1:38452)
0808 18:31:34.583031 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:34.583121 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:41793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 18:31:34.583521 950 catalog_manager.cc:4584] TS 3fc1b8efc2144d4daef20313b59d0653 (127.7.164.65:41793): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8539e5f03724fe8946f970567edafd8: Network error: Client connection negotiation failed: client connection to 127.7.164.65:41793: connect: Connection refused (error 111)
I20260808 18:31:34.589211 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 1414
I20260808 18:31:34.601329 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 926
2026-08-08T18:31:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1072 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:34 dist-test-slave-jztv krb5kdc[1657](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:34 dist-test-slave-jztv krb5kdc[1657](info): set up 2 sockets
Aug 08 18:31:34 dist-test-slave-jztv krb5kdc[1657](info): commencing operation
krb5kdc: starting...
W20260808 18:31:35.821483 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:31:35 dist-test-slave-jztv krb5kdc[1657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:31:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:31:35.953411 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:33777
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:33777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:36.070746 1675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:36.071035 1675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:36.071079 1675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:36.075204 1675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:36.075273 1675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:36.075294 1675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:36.075310 1675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:36.075325 1675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:36.080865 1675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:33777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:33777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:36.082068 1675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:36.083169 1675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:36.092124 1691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:36.092140 1686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:36.092133 1687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:36.092362 1675 server_base.cc:1034] running on GCE node
I20260808 18:31:36.093066 1675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:36.094205 1675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:36.095467 1675 hybrid_clock.cc:648] HybridClock initialized: now 1786213896095450 us; error 26 us; skew 500 ppm
Aug 08 18:31:36 dist-test-slave-jztv krb5kdc[1657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213896, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:36.098474 1675 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:31:36.099376 1675 webserver.cc:460] Webserver started at http://127.7.164.126:38599/ using document root <none> and password file <none>
I20260808 18:31:36.099917 1675 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:36.100054 1675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:36.100302 1675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:36.102491 1675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "35414fd8d18e4117baba63fde37b7b4b"
format_stamp: "Formatted at 2026-08-08 18:31:36 on dist-test-slave-jztv"
server_key: "f3e2b276e1f35ab9f670d198ebe09fed"
server_key_iv: "1fa383071085c7c70bd576bf06b805d7"
server_key_version: "encryptionkey@0"
I20260808 18:31:36.103108 1675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "35414fd8d18e4117baba63fde37b7b4b"
format_stamp: "Formatted at 2026-08-08 18:31:36 on dist-test-slave-jztv"
server_key: "f3e2b276e1f35ab9f670d198ebe09fed"
server_key_iv: "1fa383071085c7c70bd576bf06b805d7"
server_key_version: "encryptionkey@0"
I20260808 18:31:36.107040 1675 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 18:31:36.110044 1699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:36.111320 1675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 18:31:36.111461 1675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "35414fd8d18e4117baba63fde37b7b4b"
format_stamp: "Formatted at 2026-08-08 18:31:36 on dist-test-slave-jztv"
server_key: "f3e2b276e1f35ab9f670d198ebe09fed"
server_key_iv: "1fa383071085c7c70bd576bf06b805d7"
server_key_version: "encryptionkey@0"
I20260808 18:31:36.111595 1675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:36.132449 1675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:36.135560 1675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:36.135805 1675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:36.143208 1675 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:33777
I20260808 18:31:36.143283 1767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:33777 every 8 connection(s)
I20260808 18:31:36.144253 1675 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:36.147198 1768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:36.151584 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 1675
I20260808 18:31:36.151738 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:36.152027 7825 external_mini_cluster.cc:1428] Setting key d9c8985ccbd97093dc5afbb2c1cab5c7
I20260808 18:31:36.153439 1768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b: Bootstrap starting.
I20260808 18:31:36.156421 1768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:36.157398 1768 log.cc:824] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:36.159677 1768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b: No bootstrap required, opened a new log
Aug 08 18:31:36 dist-test-slave-jztv krb5kdc[1657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:31:36.164062 1768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35414fd8d18e4117baba63fde37b7b4b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33777 } }
I20260808 18:31:36.164356 1768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:36.164419 1768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35414fd8d18e4117baba63fde37b7b4b, State: Initialized, Role: FOLLOWER
I20260808 18:31:36.164928 1768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35414fd8d18e4117baba63fde37b7b4b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33777 } }
I20260808 18:31:36.165073 1768 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:36.165162 1768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:36.165274 1768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:36.166568 1768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35414fd8d18e4117baba63fde37b7b4b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33777 } }
I20260808 18:31:36.166952 1768 leader_election.cc:302] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35414fd8d18e4117baba63fde37b7b4b; no voters:
I20260808 18:31:36.167337 1768 leader_election.cc:288] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:36.167496 1773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:36.167783 1773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [term 1 LEADER]: Becoming Leader. State: Replica: 35414fd8d18e4117baba63fde37b7b4b, State: Running, Role: LEADER
I20260808 18:31:36.168140 1773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35414fd8d18e4117baba63fde37b7b4b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33777 } }
I20260808 18:31:36.168248 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.153555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42022 (local address 127.7.164.126:33777)
0808 18:31:36.154075 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:36.154096 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:36.154139 (+ 43us) server_negotiation.cc:407] Connection header received
0808 18:31:36.155078 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:36.155103 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:36.155541 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:36.155927 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:36.156787 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.158199 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.159136 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.159836 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.159999 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.162607 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:36.162613 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:36.162634 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:36.162667 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:36.164684 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:36.165172 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:36.165181 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:36.165188 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:36.165339 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:36.165640 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:36.165644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:36.165646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:36.166166 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:36.166409 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:36.166677 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 18:31:36.167188 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":147,"threads_started":1}
I20260808 18:31:36.169422 1775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35414fd8d18e4117baba63fde37b7b4b. Latest consensus state: current_term: 1 leader_uuid: "35414fd8d18e4117baba63fde37b7b4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35414fd8d18e4117baba63fde37b7b4b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33777 } } }
I20260808 18:31:36.169571 1775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:36.169911 1768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:36.170295 1776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35414fd8d18e4117baba63fde37b7b4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35414fd8d18e4117baba63fde37b7b4b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33777 } } }
I20260808 18:31:36.170387 1776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:36.171288 1781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:36.173880 1781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:36.179401 1781 catalog_manager.cc:1347] Generated new cluster ID: 96c34e45eb914ee8a1bae42ab0e81fba
I20260808 18:31:36.179473 1781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:36.197444 1781 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:36.198530 1781 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:36.208717 1781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b: Generated new TSK 0
I20260808 18:31:36.209425 1781 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:31:36.246614 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:33777
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:36.366969 1803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:36.367259 1803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:36.367332 1803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:36.371539 1803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:36.371622 1803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:36.371706 1803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:31:36.377027 1803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:33777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:36.378125 1803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:36.379189 1803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:36.389225 1814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:36.389199 1816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:36.389285 1813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:36.389438 1803 server_base.cc:1034] running on GCE node
I20260808 18:31:36.389986 1803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:36.390723 1803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:36.391916 1803 hybrid_clock.cc:648] HybridClock initialized: now 1786213896391907 us; error 29 us; skew 500 ppm
Aug 08 18:31:36 dist-test-slave-jztv krb5kdc[1657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213896, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:36.394691 1803 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:31:36.395603 1803 webserver.cc:460] Webserver started at http://127.7.164.65:35805/ using document root <none> and password file <none>
I20260808 18:31:36.396185 1803 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:36.396281 1803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:36.396528 1803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:36.398679 1803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "1c454a449bc14a88bd0d92e046ccc12e"
format_stamp: "Formatted at 2026-08-08 18:31:36 on dist-test-slave-jztv"
server_key: "ebb70c60ed5a7a3465574748f5a942af"
server_key_iv: "1f451728bb51614f08a0cd1b987f9a39"
server_key_version: "encryptionkey@0"
I20260808 18:31:36.399264 1803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "1c454a449bc14a88bd0d92e046ccc12e"
format_stamp: "Formatted at 2026-08-08 18:31:36 on dist-test-slave-jztv"
server_key: "ebb70c60ed5a7a3465574748f5a942af"
server_key_iv: "1f451728bb51614f08a0cd1b987f9a39"
server_key_version: "encryptionkey@0"
I20260808 18:31:36.403285 1803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:31:36.406543 1825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:36.407877 1803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:31:36.408025 1803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "1c454a449bc14a88bd0d92e046ccc12e"
format_stamp: "Formatted at 2026-08-08 18:31:36 on dist-test-slave-jztv"
server_key: "ebb70c60ed5a7a3465574748f5a942af"
server_key_iv: "1f451728bb51614f08a0cd1b987f9a39"
server_key_version: "encryptionkey@0"
I20260808 18:31:36.408145 1803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:36.433071 1803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:36.436218 1803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:36.436441 1803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:36.437095 1803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:36.437966 1803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:36.438058 1803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:36.438154 1803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:36.438207 1803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:36.447330 1803 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:42707
I20260808 18:31:36.447355 1977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:42707 every 8 connection(s)
I20260808 18:31:36.448274 1803 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:31:36.454840 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 1803
I20260808 18:31:36.454988 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:31:36.455319 7825 external_mini_cluster.cc:1428] Setting key c19d264ac770501e4f7d6d62df836885
Aug 08 18:31:36 dist-test-slave-jztv krb5kdc[1657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213896, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:31:36.462777 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.450062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:41047 (local address 127.7.164.126:33777)
0808 18:31:36.450244 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:36.450249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:36.451104 (+ 855us) server_negotiation.cc:407] Connection header received
0808 18:31:36.452410 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:36.452413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:36.452463 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:36.452541 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:36.453716 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.454269 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.455856 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.456136 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.456220 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.458941 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:36.458945 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:36.458948 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:36.458979 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:36.460461 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:36.461015 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:36.461018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:36.461020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:36.461117 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:36.461473 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:36.461477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:36.461479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:36.461678 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:36.461808 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:36.462535 (+ 727us) server_negotiation.cc:299] Negotiation successful
0808 18:31:36.462652 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:31:36.464126 1983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.450463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33777 (local address 127.7.164.65:41047)
0808 18:31:36.450935 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:36.450980 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:36.452213 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:36.452727 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:36.452733 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:36.453089 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:36.453586 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.453600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.454379 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.454382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.455725 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.455741 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.456291 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.456295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.456488 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.457191 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:31:36.457208 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:36.458759 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:36.460607 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:36.460613 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:36.460630 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:36.460897 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:36.461216 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:36.461219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:36.461221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:36.461369 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:36.461841 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:36.461852 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:36.462252 (+ 400us) client_negotiation.cc:769] Sending connection context
0808 18:31:36.462502 (+ 250us) client_negotiation.cc:240] Negotiation successful
0808 18:31:36.462904 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":108,"threads_started":1}
I20260808 18:31:36.465512 1978 heartbeater.cc:347] Connected to a master server at 127.7.164.126:33777
I20260808 18:31:36.465768 1978 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:31:36.466384 1978 heartbeater.cc:511] Master 127.7.164.126:33777 requested a full tablet report, sending...
I20260808 18:31:36.467957 1724 ts_manager.cc:194] Registered new tserver with Master: 1c454a449bc14a88bd0d92e046ccc12e (127.7.164.65:42707)
I20260808 18:31:36.469194 1724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:41047
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:31:36.485749 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:33777
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:36.598069 1987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:36.598403 1987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:36.598493 1987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:36.602727 1987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:36.602809 1987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:36.602898 1987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:31:36.608186 1987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:33777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:36.609253 1987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:36.610421 1987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:36.620918 1997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:36.620921 2002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:36.620954 2000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:36.621141 1987 server_base.cc:1034] running on GCE node
I20260808 18:31:36.621723 1987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:36.622460 1987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:36.623628 1987 hybrid_clock.cc:648] HybridClock initialized: now 1786213896623612 us; error 27 us; skew 500 ppm
Aug 08 18:31:36 dist-test-slave-jztv krb5kdc[1657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213896, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:36.626521 1987 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:31:36.627329 1987 webserver.cc:460] Webserver started at http://127.7.164.66:36113/ using document root <none> and password file <none>
I20260808 18:31:36.627843 1987 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:36.627910 1987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:36.628074 1987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:36.630023 1987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18"
format_stamp: "Formatted at 2026-08-08 18:31:36 on dist-test-slave-jztv"
server_key: "a4cf124f9967e0396d5f02659ed55935"
server_key_iv: "d923ba9bf202a369352b59366698eca4"
server_key_version: "encryptionkey@0"
I20260808 18:31:36.630602 1987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18"
format_stamp: "Formatted at 2026-08-08 18:31:36 on dist-test-slave-jztv"
server_key: "a4cf124f9967e0396d5f02659ed55935"
server_key_iv: "d923ba9bf202a369352b59366698eca4"
server_key_version: "encryptionkey@0"
I20260808 18:31:36.634258 1987 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:31:36.636746 2012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:36.637949 1987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:31:36.638126 1987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18"
format_stamp: "Formatted at 2026-08-08 18:31:36 on dist-test-slave-jztv"
server_key: "a4cf124f9967e0396d5f02659ed55935"
server_key_iv: "d923ba9bf202a369352b59366698eca4"
server_key_version: "encryptionkey@0"
I20260808 18:31:36.638266 1987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:36.656270 1987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:36.659793 1987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:36.660029 1987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:36.660669 1987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:36.661562 1987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:36.661634 1987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:36.661707 1987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:36.661758 1987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:36.671512 1987 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:45867
I20260808 18:31:36.671550 2162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:45867 every 8 connection(s)
I20260808 18:31:36.672581 1987 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
Aug 08 18:31:36 dist-test-slave-jztv krb5kdc[1657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213896, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:31:36.682951 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 1987
I20260808 18:31:36.683099 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:31:36.683416 7825 external_mini_cluster.cc:1428] Setting key 8ee53865b34dca134775284fb4ff731f
I20260808 18:31:36.687805 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.674512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:55347 (local address 127.7.164.126:33777)
0808 18:31:36.674639 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:36.674646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:36.675499 (+ 853us) server_negotiation.cc:407] Connection header received
0808 18:31:36.676887 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:36.676890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:36.676953 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:36.677033 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:36.678276 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.678821 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.680000 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.680228 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.680348 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.683094 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:36.683097 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:36.683099 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:36.683127 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:36.684873 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:36.685667 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:36.685670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:36.685672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:36.685764 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:36.686279 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:36.686283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:36.686284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:36.686535 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:36.686631 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:36.687409 (+ 778us) server_negotiation.cc:299] Negotiation successful
0808 18:31:36.687575 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 18:31:36.688607 2165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.674890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33777 (local address 127.7.164.66:55347)
0808 18:31:36.675348 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:36.675393 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:36.676670 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:36.677213 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:36.677220 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:36.677591 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:36.678096 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.678111 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.678966 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.678969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.679863 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.679873 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.680394 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.680398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.680574 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.681206 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:31:36.681224 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:36.682887 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:36.685018 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:36.685024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:36.685042 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:36.685463 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:36.685874 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:36.685879 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:36.685884 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:36.686099 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:36.686661 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:36.686671 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:36.687047 (+ 376us) client_negotiation.cc:769] Sending connection context
0808 18:31:36.687278 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 18:31:36.687662 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":113,"threads_started":1}
I20260808 18:31:36.690047 2163 heartbeater.cc:347] Connected to a master server at 127.7.164.126:33777
I20260808 18:31:36.690416 2163 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:36.691110 2163 heartbeater.cc:511] Master 127.7.164.126:33777 requested a full tablet report, sending...
I20260808 18:31:36.692548 1724 ts_manager.cc:194] Registered new tserver with Master: 0c823bdabc3f4fe8ba2f0ea236f6ef18 (127.7.164.66:45867)
I20260808 18:31:36.693445 1724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:55347
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:31:36.717733 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:33777
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:44665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:36.831957 2170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:36.832212 2170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:36.832257 2170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:36.836318 2170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:36.836371 2170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:36.836428 2170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:31:36.841634 2170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:44665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:33777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:36.842703 2170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:36.843694 2170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:36.852922 2185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:36.852918 2181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:36.852929 2180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:36.853350 2170 server_base.cc:1034] running on GCE node
I20260808 18:31:36.853739 2170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:36.854401 2170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:36.855572 2170 hybrid_clock.cc:648] HybridClock initialized: now 1786213896855553 us; error 32 us; skew 500 ppm
Aug 08 18:31:36 dist-test-slave-jztv krb5kdc[1657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213896, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:36.858419 2170 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:31:36.859376 2170 webserver.cc:460] Webserver started at http://127.7.164.67:46813/ using document root <none> and password file <none>
I20260808 18:31:36.859936 2170 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:36.860000 2170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:36.860246 2170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:36.862265 2170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "fc70c48ad78249e2aac5f911257cd5cc"
format_stamp: "Formatted at 2026-08-08 18:31:36 on dist-test-slave-jztv"
server_key: "5e104357d74aba7a8307e12c320b781f"
server_key_iv: "253100bb81a5b44e01e0c5ddf9b86747"
server_key_version: "encryptionkey@0"
I20260808 18:31:36.862855 2170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "fc70c48ad78249e2aac5f911257cd5cc"
format_stamp: "Formatted at 2026-08-08 18:31:36 on dist-test-slave-jztv"
server_key: "5e104357d74aba7a8307e12c320b781f"
server_key_iv: "253100bb81a5b44e01e0c5ddf9b86747"
server_key_version: "encryptionkey@0"
I20260808 18:31:36.866444 2170 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:31:36.868748 2194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:36.870019 2170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:31:36.870155 2170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "fc70c48ad78249e2aac5f911257cd5cc"
format_stamp: "Formatted at 2026-08-08 18:31:36 on dist-test-slave-jztv"
server_key: "5e104357d74aba7a8307e12c320b781f"
server_key_iv: "253100bb81a5b44e01e0c5ddf9b86747"
server_key_version: "encryptionkey@0"
I20260808 18:31:36.870337 2170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:36.883725 2170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:36.886829 2170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:36.887024 2170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:36.887574 2170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:36.888363 2170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:36.888429 2170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:36.888500 2170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:36.888553 2170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:36.897163 2170 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:33395
I20260808 18:31:36.897203 2344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:33395 every 8 connection(s)
I20260808 18:31:36.897980 2170 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:31:36.904826 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 2170
I20260808 18:31:36.904992 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:31:36.905294 7825 external_mini_cluster.cc:1428] Setting key 743a697dfd609050a92dcb0618215235
Aug 08 18:31:36 dist-test-slave-jztv krb5kdc[1657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213896, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:31:36.911011 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.899688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:38907 (local address 127.7.164.126:33777)
0808 18:31:36.899824 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:36.899828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:36.900573 (+ 745us) server_negotiation.cc:407] Connection header received
0808 18:31:36.901466 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:36.901470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:36.901528 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:36.901753 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:36.902917 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.903453 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.904606 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.904820 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.904945 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.907524 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:36.907528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:36.907531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:36.907562 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:36.908970 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:36.909481 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:36.909484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:36.909486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:36.909572 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:36.909919 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:36.909922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:36.909924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:36.910105 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:36.910212 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:36.910715 (+ 503us) server_negotiation.cc:299] Negotiation successful
0808 18:31:36.910834 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:31:36.911720 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.900016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33777 (local address 127.7.164.67:38907)
0808 18:31:36.900434 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:36.900473 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:36.901263 (+ 790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:36.901900 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:36.901907 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:36.902308 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:36.902781 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.902794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.903609 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.903612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.904491 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.904500 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.904990 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.904995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.905247 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.905890 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:31:36.905905 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:36.907360 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:36.909101 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:36.909106 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:36.909119 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:36.909371 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:36.909666 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:36.909669 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:36.909671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:36.909810 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:36.910211 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:36.910242 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:36.910476 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 18:31:36.910672 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 18:31:36.910965 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":108,"threads_started":1}
I20260808 18:31:36.912799 2346 heartbeater.cc:347] Connected to a master server at 127.7.164.126:33777
I20260808 18:31:36.913045 2346 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:36.913522 2346 heartbeater.cc:511] Master 127.7.164.126:33777 requested a full tablet report, sending...
I20260808 18:31:36.914590 1724 ts_manager.cc:194] Registered new tserver with Master: fc70c48ad78249e2aac5f911257cd5cc (127.7.164.67:33395)
I20260808 18:31:36.915272 1724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:38907
I20260808 18:31:36.919003 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:31:36.928303 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.920939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42028 (local address 127.7.164.126:33777)
0808 18:31:36.921074 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:36.921077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:36.921266 (+ 189us) server_negotiation.cc:407] Connection header received
0808 18:31:36.921352 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:36.921354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:36.921389 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:36.921465 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:36.922144 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.922699 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.923490 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.923722 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.923799 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.925280 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:36.925283 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:36.925285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:36.925305 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:36.926684 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:36.927199 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:36.927201 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:36.927203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:36.927274 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:36.927552 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:36.927554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:36.927555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:36.927743 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:36.927853 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:36.928121 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 18:31:36.928196 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:31:36.930969 1724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:31:36.933779 1724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:31:36.949029 2364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.942638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:33395 (local address 127.0.0.1:43938)
0808 18:31:36.943541 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:36.943555 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:36.943663 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:36.944363 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:36.944369 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:36.944393 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:36.944563 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.944571 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.946124 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.946128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.947605 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.947618 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.948471 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.948475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.948584 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.948621 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 18:31:36.948735 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 18:31:36.948854 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":129,"threads_started":1}
I20260808 18:31:36.949162 2362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.942415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:42707 (local address 127.0.0.1:38296)
0808 18:31:36.943254 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:36.943309 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:36.943447 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:36.944580 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:36.944583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:36.944615 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:36.944805 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.944810 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.946301 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.946304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.947686 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.947697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.948750 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.948753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.948836 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.948857 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:31:36.948934 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 18:31:36.948989 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":308,"spinlock_wait_cycles":5120,"thread_start_us":90,"threads_started":1}
I20260808 18:31:36.949571 2363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.942545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38296 (local address 127.7.164.65:42707)
0808 18:31:36.944035 (+ 1490us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:36.944051 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:36.944071 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:31:36.944134 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:36.944139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:36.944294 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:36.944423 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:36.944934 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.946053 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.947813 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.948634 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.948740 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.948872 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:36.948961 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 18:31:36.949054 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1393,"thread_start_us":75,"threads_started":1}
I20260808 18:31:36.949582 2368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.942822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43938 (local address 127.7.164.67:33395)
0808 18:31:36.943291 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:36.943307 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:36.943899 (+ 592us) server_negotiation.cc:407] Connection header received
0808 18:31:36.943986 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:36.943992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:36.944197 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:36.944362 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:36.944706 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.945978 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.947759 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.948330 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.948471 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.948622 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:36.949378 (+ 756us) server_negotiation.cc:299] Negotiation successful
0808 18:31:36.949472 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":125,"threads_started":1}
I20260808 18:31:36.950701 2369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.943206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:45867 (local address 127.0.0.1:55674)
0808 18:31:36.943703 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:36.943716 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:36.943800 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:36.944580 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:36.944583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:36.944613 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:36.944802 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.944809 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.947321 (+ 2512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.947328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.948366 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.948378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.950283 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.950287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.950373 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.950397 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:31:36.950468 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 18:31:36.950554 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":40,"spinlock_wait_cycles":5248,"thread_start_us":84,"threads_started":1}
I20260808 18:31:36.951118 2371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.943746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55674 (local address 127.7.164.66:45867)
0808 18:31:36.944150 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:36.944159 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:36.944178 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:31:36.944239 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:36.944243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:36.944435 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:36.944586 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:36.946253 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.947186 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.948532 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.949959 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.950065 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.950192 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:36.950900 (+ 708us) server_negotiation.cc:299] Negotiation successful
0808 18:31:36.951000 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":130,"threads_started":1}
I20260808 18:31:36.952919 1889 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9834c764764e2887a1fe9bf8f7c789 (DEFAULT_TABLE table=test-table [id=eb1bcbc62e324a8799b7882fa906cfd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:36.952919 2259 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9834c764764e2887a1fe9bf8f7c789 (DEFAULT_TABLE table=test-table [id=eb1bcbc62e324a8799b7882fa906cfd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:36.953230 2078 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9834c764764e2887a1fe9bf8f7c789 (DEFAULT_TABLE table=test-table [id=eb1bcbc62e324a8799b7882fa906cfd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:36.954077 1889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f9834c764764e2887a1fe9bf8f7c789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:36.954082 2259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f9834c764764e2887a1fe9bf8f7c789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:36.954221 2078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f9834c764764e2887a1fe9bf8f7c789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:36.960083 2372 tablet_bootstrap.cc:492] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18: Bootstrap starting.
I20260808 18:31:36.961082 2374 tablet_bootstrap.cc:492] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc: Bootstrap starting.
I20260808 18:31:36.962342 2375 tablet_bootstrap.cc:492] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e: Bootstrap starting.
I20260808 18:31:36.963307 2374 tablet_bootstrap.cc:654] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:36.963639 2372 tablet_bootstrap.cc:654] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:36.964218 2375 tablet_bootstrap.cc:654] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:36.964607 2372 log.cc:824] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:36.964607 2374 log.cc:824] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:36.965355 2375 log.cc:824] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:36.966665 2372 tablet_bootstrap.cc:492] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18: No bootstrap required, opened a new log
I20260808 18:31:36.966763 2374 tablet_bootstrap.cc:492] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc: No bootstrap required, opened a new log
I20260808 18:31:36.966921 2372 ts_tablet_manager.cc:1397] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 18:31:36.966961 2374 ts_tablet_manager.cc:1397] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:31:36.968253 2375 tablet_bootstrap.cc:492] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e: No bootstrap required, opened a new log
I20260808 18:31:36.968438 2375 ts_tablet_manager.cc:1397] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:31:36.970033 2374 raft_consensus.cc:348] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c454a449bc14a88bd0d92e046ccc12e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42707 } } peers { permanent_uuid: "fc70c48ad78249e2aac5f911257cd5cc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 33395 } } peers { permanent_uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45867 } }
I20260808 18:31:36.970363 2374 raft_consensus.cc:374] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:36.970424 2374 raft_consensus.cc:729] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc70c48ad78249e2aac5f911257cd5cc, State: Initialized, Role: FOLLOWER
I20260808 18:31:36.970938 2374 consensus_queue.cc:260] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c454a449bc14a88bd0d92e046ccc12e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42707 } } peers { permanent_uuid: "fc70c48ad78249e2aac5f911257cd5cc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 33395 } } peers { permanent_uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45867 } }
I20260808 18:31:36.970854 2372 raft_consensus.cc:348] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c454a449bc14a88bd0d92e046ccc12e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42707 } } peers { permanent_uuid: "fc70c48ad78249e2aac5f911257cd5cc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 33395 } } peers { permanent_uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45867 } }
I20260808 18:31:36.971120 2372 raft_consensus.cc:374] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:36.971186 2372 raft_consensus.cc:729] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c823bdabc3f4fe8ba2f0ea236f6ef18, State: Initialized, Role: FOLLOWER
I20260808 18:31:36.971632 2372 consensus_queue.cc:260] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c454a449bc14a88bd0d92e046ccc12e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42707 } } peers { permanent_uuid: "fc70c48ad78249e2aac5f911257cd5cc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 33395 } } peers { permanent_uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45867 } }
I20260808 18:31:36.972034 2374 ts_tablet_manager.cc:1428] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:31:36.972510 2346 heartbeater.cc:503] Master 127.7.164.126:33777 was elected leader, sending a full tablet report...
I20260808 18:31:36.972369 2375 raft_consensus.cc:348] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c454a449bc14a88bd0d92e046ccc12e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42707 } } peers { permanent_uuid: "fc70c48ad78249e2aac5f911257cd5cc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 33395 } } peers { permanent_uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45867 } }
I20260808 18:31:36.972558 2372 ts_tablet_manager.cc:1428] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:31:36.972617 2375 raft_consensus.cc:374] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:36.972666 2375 raft_consensus.cc:729] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c454a449bc14a88bd0d92e046ccc12e, State: Initialized, Role: FOLLOWER
I20260808 18:31:36.973035 2375 consensus_queue.cc:260] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c454a449bc14a88bd0d92e046ccc12e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42707 } } peers { permanent_uuid: "fc70c48ad78249e2aac5f911257cd5cc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 33395 } } peers { permanent_uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45867 } }
I20260808 18:31:36.973376 2385 raft_consensus.cc:482] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:31:36.973553 2385 raft_consensus.cc:504] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c454a449bc14a88bd0d92e046ccc12e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42707 } } peers { permanent_uuid: "fc70c48ad78249e2aac5f911257cd5cc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 33395 } } peers { permanent_uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45867 } }
I20260808 18:31:36.973649 2163 heartbeater.cc:503] Master 127.7.164.126:33777 was elected leader, sending a full tablet report...
I20260808 18:31:36.973958 2375 ts_tablet_manager.cc:1428] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:31:36.974786 1978 heartbeater.cc:503] Master 127.7.164.126:33777 was elected leader, sending a full tablet report...
I20260808 18:31:36.975711 2385 leader_election.cc:288] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c454a449bc14a88bd0d92e046ccc12e (127.7.164.65:42707), 0c823bdabc3f4fe8ba2f0ea236f6ef18 (127.7.164.66:45867)
I20260808 18:31:36.980085 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.975012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:42707 (local address 127.7.164.67:35207)
0808 18:31:36.975272 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:36.975287 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:36.975366 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:36.976389 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:36.976392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:36.976410 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:36.976535 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.976546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.977561 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.977565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.978778 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.978790 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.979560 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.979564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.979752 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.979770 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 18:31:36.979867 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 18:31:36.979944 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":56}
I20260808 18:31:36.980752 2363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.975520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:35207 (local address 127.7.164.65:42707)
0808 18:31:36.976167 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:36.976173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:36.976187 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:31:36.976234 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:36.976237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:36.976283 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:36.976376 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:36.976681 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.977432 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.978890 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.979432 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.979571 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.979630 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:36.980612 (+ 982us) server_negotiation.cc:299] Negotiation successful
0808 18:31:36.980670 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568}
I20260808 18:31:36.980942 2371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.976113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:57103 (local address 127.7.164.66:45867)
0808 18:31:36.976217 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:36.976221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:36.976551 (+ 330us) server_negotiation.cc:407] Connection header received
0808 18:31:36.976742 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:36.976745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:36.976796 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:36.976850 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:36.977711 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.978400 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.979880 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.980441 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.980532 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.980565 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:36.980774 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 18:31:36.980843 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:31:36.980966 2387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:36.976035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:45867 (local address 127.7.164.67:57103)
0808 18:31:36.976446 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:36.976462 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:36.976622 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:36.977430 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:36.977433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:36.977448 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:36.977565 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.977573 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.978537 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.978539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.979750 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:36.979762 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:36.980567 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:36.980570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:36.980642 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:36.980662 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 18:31:36.980755 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 18:31:36.980819 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":9,"thread_start_us":102,"threads_started":1}
I20260808 18:31:36.981312 1914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9834c764764e2887a1fe9bf8f7c789" candidate_uuid: "fc70c48ad78249e2aac5f911257cd5cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c454a449bc14a88bd0d92e046ccc12e" is_pre_election: true
I20260808 18:31:36.981474 2103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9834c764764e2887a1fe9bf8f7c789" candidate_uuid: "fc70c48ad78249e2aac5f911257cd5cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18" is_pre_election: true
I20260808 18:31:36.981616 1914 raft_consensus.cc:2454] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc70c48ad78249e2aac5f911257cd5cc in term 0.
I20260808 18:31:36.981678 2103 raft_consensus.cc:2454] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc70c48ad78249e2aac5f911257cd5cc in term 0.
I20260808 18:31:36.982127 2198 leader_election.cc:302] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0c823bdabc3f4fe8ba2f0ea236f6ef18, 1c454a449bc14a88bd0d92e046ccc12e, fc70c48ad78249e2aac5f911257cd5cc; no voters:
I20260808 18:31:36.982438 2383 raft_consensus.cc:2781] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:31:36.982540 2383 raft_consensus.cc:482] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:31:36.982600 2383 raft_consensus.cc:3037] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:36.983594 2383 raft_consensus.cc:504] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c454a449bc14a88bd0d92e046ccc12e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42707 } } peers { permanent_uuid: "fc70c48ad78249e2aac5f911257cd5cc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 33395 } } peers { permanent_uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45867 } }
I20260808 18:31:36.983932 2383 leader_election.cc:288] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [CANDIDATE]: Term 1 election: Requested vote from peers 1c454a449bc14a88bd0d92e046ccc12e (127.7.164.65:42707), 0c823bdabc3f4fe8ba2f0ea236f6ef18 (127.7.164.66:45867)
I20260808 18:31:36.984282 2103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9834c764764e2887a1fe9bf8f7c789" candidate_uuid: "fc70c48ad78249e2aac5f911257cd5cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18"
I20260808 18:31:36.984432 2103 raft_consensus.cc:3037] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:36.985419 2103 raft_consensus.cc:2454] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc70c48ad78249e2aac5f911257cd5cc in term 1.
I20260808 18:31:36.985700 1914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9834c764764e2887a1fe9bf8f7c789" candidate_uuid: "fc70c48ad78249e2aac5f911257cd5cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c454a449bc14a88bd0d92e046ccc12e"
I20260808 18:31:36.985733 2198 leader_election.cc:302] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c823bdabc3f4fe8ba2f0ea236f6ef18, fc70c48ad78249e2aac5f911257cd5cc; no voters:
I20260808 18:31:36.985828 1914 raft_consensus.cc:3037] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:36.985945 2383 raft_consensus.cc:2781] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:36.986047 2383 raft_consensus.cc:686] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [term 1 LEADER]: Becoming Leader. State: Replica: fc70c48ad78249e2aac5f911257cd5cc, State: Running, Role: LEADER
I20260808 18:31:36.986404 2383 consensus_queue.cc:237] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c454a449bc14a88bd0d92e046ccc12e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42707 } } peers { permanent_uuid: "fc70c48ad78249e2aac5f911257cd5cc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 33395 } } peers { permanent_uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45867 } }
I20260808 18:31:36.987169 1914 raft_consensus.cc:2454] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc70c48ad78249e2aac5f911257cd5cc in term 1.
I20260808 18:31:36.989887 1724 catalog_manager.cc:5495] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc reported cstate change: term changed from 0 to 1, leader changed from <none> to fc70c48ad78249e2aac5f911257cd5cc (127.7.164.67). New cstate: current_term: 1 leader_uuid: "fc70c48ad78249e2aac5f911257cd5cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c454a449bc14a88bd0d92e046ccc12e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc70c48ad78249e2aac5f911257cd5cc" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 33395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45867 } health_report { overall_health: UNKNOWN } } }
I20260808 18:31:37.007193 2368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.003786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43940 (local address 127.7.164.67:33395)
0808 18:31:37.003918 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:37.003921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:37.003938 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:31:37.004003 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:37.004005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:37.004059 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:37.004146 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:31:37.004382 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.004882 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.005732 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.005963 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.006055 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.006429 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:31:37.006840 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:31:37.006961 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:37.007042 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 18:31:37.007074 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:31:37.013208 2103 raft_consensus.cc:1261] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18 [term 1 FOLLOWER]: Refusing update from remote peer fc70c48ad78249e2aac5f911257cd5cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:31:37.013207 1914 raft_consensus.cc:1261] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e [term 1 FOLLOWER]: Refusing update from remote peer fc70c48ad78249e2aac5f911257cd5cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:31:37.013970 2383 consensus_queue.cc:1035] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c823bdabc3f4fe8ba2f0ea236f6ef18" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 45867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:31:37.014191 2385 consensus_queue.cc:1035] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c454a449bc14a88bd0d92e046ccc12e" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 42707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:31:37.022845 2393 mvcc.cc:205] Tried to move back new op lower bound from 7316332122158759936 to 7316332122059505664. Current Snapshot: MvccSnapshot[applied={T|T < 7316332122158759936}]
I20260808 18:31:37.031257 1724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:31:37.031412 1724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:31:37.033888 1724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35414fd8d18e4117baba63fde37b7b4b: Sending DeleteTablet for 3 replicas of tablet 1f9834c764764e2887a1fe9bf8f7c789
I20260808 18:31:37.034722 1889 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f9834c764764e2887a1fe9bf8f7c789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:37 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:38296
I20260808 18:31:37.034888 2078 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f9834c764764e2887a1fe9bf8f7c789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:37 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:55674
I20260808 18:31:37.035310 2403 tablet_replica.cc:330] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e: stopping tablet replica
I20260808 18:31:37.035315 2259 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f9834c764764e2887a1fe9bf8f7c789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:37 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:43938
I20260808 18:31:37.035432 2404 tablet_replica.cc:330] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18: stopping tablet replica
I20260808 18:31:37.035744 2403 raft_consensus.cc:2229] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:31:37.035748 2404 raft_consensus.cc:2229] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:31:37.036134 2403 raft_consensus.cc:2258] T 1f9834c764764e2887a1fe9bf8f7c789 P 1c454a449bc14a88bd0d92e046ccc12e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:31:37.036134 2404 raft_consensus.cc:2258] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:31:37.036604 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 1803
I20260808 18:31:37.037932 2404 ts_tablet_manager.cc:1905] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:31:37.042107 2404 ts_tablet_manager.cc:1918] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:31:37.042263 2404 log.cc:1196] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/1f9834c764764e2887a1fe9bf8f7c789
I20260808 18:31:37.042719 2404 ts_tablet_manager.cc:1939] T 1f9834c764764e2887a1fe9bf8f7c789 P 0c823bdabc3f4fe8ba2f0ea236f6ef18: Deleting consensus metadata
I20260808 18:31:37.044682 1701 catalog_manager.cc:4841] TS 0c823bdabc3f4fe8ba2f0ea236f6ef18 (127.7.164.66:45867): tablet 1f9834c764764e2887a1fe9bf8f7c789 (table test-table [id=eb1bcbc62e324a8799b7882fa906cfd0]) successfully deleted
I20260808 18:31:37.050568 2405 tablet_replica.cc:330] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc: stopping tablet replica
I20260808 18:31:37.050915 2405 raft_consensus.cc:2229] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:31:37.051378 2405 raft_consensus.cc:2258] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:31:37.052987 2405 ts_tablet_manager.cc:1905] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 18:31:37.054672 1703 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:42707 (error 108)
I20260808 18:31:37.055608 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 1987
I20260808 18:31:37.055843 2369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.055475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:42707 (local address 127.0.0.1:38302)
0808 18:31:37.055641 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:37.055736 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 18:31:37.062940 1703 catalog_manager.cc:4584] TS 1c454a449bc14a88bd0d92e046ccc12e (127.7.164.65:42707): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1f9834c764764e2887a1fe9bf8f7c789: Network error: Client connection negotiation failed: client connection to 127.7.164.65:42707: connect: Connection refused (error 111)
I20260808 18:31:37.064041 2405 ts_tablet_manager.cc:1918] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:31:37.064183 2405 log.cc:1196] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/1f9834c764764e2887a1fe9bf8f7c789
I20260808 18:31:37.064586 2405 ts_tablet_manager.cc:1939] T 1f9834c764764e2887a1fe9bf8f7c789 P fc70c48ad78249e2aac5f911257cd5cc: Deleting consensus metadata
I20260808 18:31:37.066157 1701 catalog_manager.cc:4841] TS fc70c48ad78249e2aac5f911257cd5cc (127.7.164.67:33395): tablet 1f9834c764764e2887a1fe9bf8f7c789 (table test-table [id=eb1bcbc62e324a8799b7882fa906cfd0]) successfully deleted
I20260808 18:31:37.069294 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 2170
I20260808 18:31:37.088841 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 1675
2026-08-08T18:31:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2488 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T18:31:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:37Z Disabled control of system clock
I20260808 18:31:37.127897 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:37283
--webserver_interface=127.7.164.126
--webserver_port=0
--builtin_ntp_servers=127.7.164.84:46061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:37283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:31:37.240154 2413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:37.240434 2413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:37.240470 2413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:37.244683 2413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:37.244751 2413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:37.244766 2413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:37.244783 2413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:37.244799 2413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:37.249653 2413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:46061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:37283
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:37283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:37.250835 2413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:37.252223 2413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:37.261106 2420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:37.261147 2424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:37.261142 2421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:37.261339 2413 server_base.cc:1034] running on GCE node
I20260808 18:31:37.262022 2413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:37.263175 2413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:37.264403 2413 hybrid_clock.cc:648] HybridClock initialized: now 1786213897264372 us; error 50 us; skew 500 ppm
I20260808 18:31:37.266390 2413 webserver.cc:460] Webserver started at http://127.7.164.126:37153/ using document root <none> and password file <none>
I20260808 18:31:37.266983 2413 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:37.267077 2413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:37.267319 2413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:37.269381 2413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "a861606983154ebd8e74efd128399f6e"
format_stamp: "Formatted at 2026-08-08 18:31:37 on dist-test-slave-jztv"
server_key: "f05c4398ccbc1eafce3c26e54445b9a6"
server_key_iv: "aa9f34aec447ab89966a767b5c5491f2"
server_key_version: "encryptionkey@0"
I20260808 18:31:37.269965 2413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "a861606983154ebd8e74efd128399f6e"
format_stamp: "Formatted at 2026-08-08 18:31:37 on dist-test-slave-jztv"
server_key: "f05c4398ccbc1eafce3c26e54445b9a6"
server_key_iv: "aa9f34aec447ab89966a767b5c5491f2"
server_key_version: "encryptionkey@0"
I20260808 18:31:37.273679 2413 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:31:37.276113 2431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:37.277410 2413 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:31:37.277576 2413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "a861606983154ebd8e74efd128399f6e"
format_stamp: "Formatted at 2026-08-08 18:31:37 on dist-test-slave-jztv"
server_key: "f05c4398ccbc1eafce3c26e54445b9a6"
server_key_iv: "aa9f34aec447ab89966a767b5c5491f2"
server_key_version: "encryptionkey@0"
I20260808 18:31:37.277700 2413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:37.290617 2413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:37.291296 2413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:37.291509 2413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:37.298974 2413 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:37283
I20260808 18:31:37.298995 2499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:37283 every 8 connection(s)
I20260808 18:31:37.299908 2413 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:37.303030 2500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:37.304178 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 2413
I20260808 18:31:37.304275 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:37.304611 7825 external_mini_cluster.cc:1428] Setting key da7669b2e6963485e4160ccf6e6f938c
I20260808 18:31:37.309620 2500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e: Bootstrap starting.
I20260808 18:31:37.312482 2500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:37.313267 2500 log.cc:824] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:37.314375 2503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.305900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42530 (local address 127.7.164.126:37283)
0808 18:31:37.306566 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:37.306580 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:37.306615 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:31:37.307286 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:37.307304 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:37.307693 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:37.308025 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:37.308451 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.309402 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.310742 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.311211 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.311392 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.312319 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:37.312324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:37.312337 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:37.312772 (+ 435us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:37.312827 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:37.312848 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:37.312963 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:37.313136 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 18:31:37.313430 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":117,"threads_started":1}
I20260808 18:31:37.315420 2500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e: No bootstrap required, opened a new log
I20260808 18:31:37.319496 2500 raft_consensus.cc:348] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a861606983154ebd8e74efd128399f6e" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37283 } }
I20260808 18:31:37.319808 2500 raft_consensus.cc:374] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:37.319877 2500 raft_consensus.cc:729] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a861606983154ebd8e74efd128399f6e, State: Initialized, Role: FOLLOWER
I20260808 18:31:37.320313 2500 consensus_queue.cc:260] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a861606983154ebd8e74efd128399f6e" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37283 } }
I20260808 18:31:37.320443 2500 raft_consensus.cc:388] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:37.320490 2500 raft_consensus.cc:482] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:37.320564 2500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:37.321504 2500 raft_consensus.cc:504] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a861606983154ebd8e74efd128399f6e" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37283 } }
I20260808 18:31:37.321798 2500 leader_election.cc:302] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a861606983154ebd8e74efd128399f6e; no voters:
I20260808 18:31:37.322124 2500 leader_election.cc:288] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:37.322271 2505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:37.322527 2505 raft_consensus.cc:686] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [term 1 LEADER]: Becoming Leader. State: Replica: a861606983154ebd8e74efd128399f6e, State: Running, Role: LEADER
I20260808 18:31:37.322861 2505 consensus_queue.cc:237] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a861606983154ebd8e74efd128399f6e" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37283 } }
I20260808 18:31:37.323441 2500 sys_catalog.cc:564] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:37.324640 2509 sys_catalog.cc:455] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a861606983154ebd8e74efd128399f6e. Latest consensus state: current_term: 1 leader_uuid: "a861606983154ebd8e74efd128399f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a861606983154ebd8e74efd128399f6e" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37283 } } }
I20260808 18:31:37.324640 2508 sys_catalog.cc:455] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a861606983154ebd8e74efd128399f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a861606983154ebd8e74efd128399f6e" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37283 } } }
I20260808 18:31:37.324803 2509 sys_catalog.cc:458] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:37.324813 2508 sys_catalog.cc:458] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:37.325281 2520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:37.328094 2520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:37.334188 2520 catalog_manager.cc:1347] Generated new cluster ID: 9f7b76ca997f47d89e68874b81ad3dd5
I20260808 18:31:37.334301 2520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:37.346838 2520 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:37.347788 2520 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:37.354923 2520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e: Generated new TSK 0
I20260808 18:31:37.355616 2520 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:31:37.364210 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:37283
--builtin_ntp_servers=127.7.164.84:46061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:31:37.483397 2539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:37.483695 2539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:37.483734 2539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:37.487872 2539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:37.487936 2539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:37.487993 2539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:31:37.492897 2539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:46061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:37283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:37.493999 2539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:37.495451 2539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:37.505403 2550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:37.505403 2552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:37.505453 2548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:37.505910 2539 server_base.cc:1034] running on GCE node
I20260808 18:31:37.506426 2539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:37.507121 2539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:37.508330 2539 hybrid_clock.cc:648] HybridClock initialized: now 1786213897508291 us; error 52 us; skew 500 ppm
I20260808 18:31:37.510334 2539 webserver.cc:460] Webserver started at http://127.7.164.65:44193/ using document root <none> and password file <none>
I20260808 18:31:37.510946 2539 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:37.511042 2539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:37.511294 2539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:37.513528 2539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "6ad9268bd0134d10aef05979c25ac318"
format_stamp: "Formatted at 2026-08-08 18:31:37 on dist-test-slave-jztv"
server_key: "eb264e5872a94f54e566a5e32c2b0303"
server_key_iv: "e12b0c72373d9db9c143192a1547d6ed"
server_key_version: "encryptionkey@0"
I20260808 18:31:37.514168 2539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "6ad9268bd0134d10aef05979c25ac318"
format_stamp: "Formatted at 2026-08-08 18:31:37 on dist-test-slave-jztv"
server_key: "eb264e5872a94f54e566a5e32c2b0303"
server_key_iv: "e12b0c72373d9db9c143192a1547d6ed"
server_key_version: "encryptionkey@0"
I20260808 18:31:37.519399 2539 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 18:31:37.522020 2560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:37.523489 2539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:31:37.523624 2539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "6ad9268bd0134d10aef05979c25ac318"
format_stamp: "Formatted at 2026-08-08 18:31:37 on dist-test-slave-jztv"
server_key: "eb264e5872a94f54e566a5e32c2b0303"
server_key_iv: "e12b0c72373d9db9c143192a1547d6ed"
server_key_version: "encryptionkey@0"
I20260808 18:31:37.523762 2539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:37.545007 2539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:37.545794 2539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:37.546011 2539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:37.546739 2539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:37.547744 2539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:37.547820 2539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:37.547891 2539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:37.547926 2539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:37.556689 2539 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:38395
I20260808 18:31:37.556720 2705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:38395 every 8 connection(s)
I20260808 18:31:37.557657 2539 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:31:37.562016 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 2539
I20260808 18:31:37.562160 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:31:37.562498 7825 external_mini_cluster.cc:1428] Setting key c10c64725883657ecf4c8fc906012929
I20260808 18:31:37.564631 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:37283
--builtin_ntp_servers=127.7.164.84:46061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:31:37.568572 2503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.559627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:42581 (local address 127.7.164.126:37283)
0808 18:31:37.559781 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:37.559785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:37.560907 (+ 1122us) server_negotiation.cc:407] Connection header received
0808 18:31:37.562481 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:37.562484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:37.562549 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:37.562663 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:37.563809 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.564430 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.566293 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.566693 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.566824 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.568028 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:37.568034 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:37.568040 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:37.568065 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:37.568090 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:37.568104 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:37.568154 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:37.568396 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 18:31:37.568453 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:31:37.569798 2708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.560068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37283 (local address 127.7.164.65:42581)
0808 18:31:37.560751 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:37.560792 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:37.562176 (+ 1384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:37.562903 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:37.562914 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:37.563030 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:37.563653 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:37.563667 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.564584 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.564588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:37.566088 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:37.566104 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.566868 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.566871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:37.567020 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.567707 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:31:37.567708 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:37.567750 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:31:37.567816 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:37.568228 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:37.568240 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:37.568249 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 18:31:37.568374 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 18:31:37.568687 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":121,"threads_started":1}
I20260808 18:31:37.571170 2706 heartbeater.cc:347] Connected to a master server at 127.7.164.126:37283
I20260808 18:31:37.571417 2706 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:37.571941 2706 heartbeater.cc:511] Master 127.7.164.126:37283 requested a full tablet report, sending...
I20260808 18:31:37.573472 2457 ts_manager.cc:194] Registered new tserver with Master: 6ad9268bd0134d10aef05979c25ac318 (127.7.164.65:38395)
I20260808 18:31:37.574717 2457 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.164.65:42581
W20260808 18:31:37.682433 2709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:37.682806 2709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:37.682888 2709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:37.687649 2709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:37.687758 2709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:37.687861 2709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:31:37.692798 2709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:46061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:37283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:37.693933 2709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:37.695365 2709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:37.705173 2716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:37.705184 2715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:37.705281 2709 server_base.cc:1034] running on GCE node
W20260808 18:31:37.705188 2719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:37.705834 2709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:37.706561 2709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:37.707748 2709 hybrid_clock.cc:648] HybridClock initialized: now 1786213897707709 us; error 55 us; skew 500 ppm
I20260808 18:31:37.709558 2709 webserver.cc:460] Webserver started at http://127.7.164.66:44221/ using document root <none> and password file <none>
I20260808 18:31:37.710130 2709 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:37.710242 2709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:37.710485 2709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:37.712618 2709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "0a1f2712c5af4bd7bc13c68a8f80df93"
format_stamp: "Formatted at 2026-08-08 18:31:37 on dist-test-slave-jztv"
server_key: "df79476a8bfeb3299a1697a2cc7164cc"
server_key_iv: "dce59cc5c4da4b5613a44144bdd4c650"
server_key_version: "encryptionkey@0"
I20260808 18:31:37.713326 2709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "0a1f2712c5af4bd7bc13c68a8f80df93"
format_stamp: "Formatted at 2026-08-08 18:31:37 on dist-test-slave-jztv"
server_key: "df79476a8bfeb3299a1697a2cc7164cc"
server_key_iv: "dce59cc5c4da4b5613a44144bdd4c650"
server_key_version: "encryptionkey@0"
I20260808 18:31:37.717128 2709 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:31:37.719722 2725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:37.720883 2709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:31:37.721028 2709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "0a1f2712c5af4bd7bc13c68a8f80df93"
format_stamp: "Formatted at 2026-08-08 18:31:37 on dist-test-slave-jztv"
server_key: "df79476a8bfeb3299a1697a2cc7164cc"
server_key_iv: "dce59cc5c4da4b5613a44144bdd4c650"
server_key_version: "encryptionkey@0"
I20260808 18:31:37.721145 2709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:37.735565 2709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:37.736227 2709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:37.736421 2709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:37.736992 2709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:37.737928 2709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:37.737998 2709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:37.738066 2709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:37.738116 2709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:37.746856 2709 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:41237
I20260808 18:31:37.746886 2886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:41237 every 8 connection(s)
I20260808 18:31:37.747715 2709 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:31:37.752997 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 2709
I20260808 18:31:37.753130 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:31:37.753448 7825 external_mini_cluster.cc:1428] Setting key f5536d40a1d49903b03cbd88e65b4ee6
I20260808 18:31:37.755582 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:37283
--builtin_ntp_servers=127.7.164.84:46061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:31:37.757671 2503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.749516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:55215 (local address 127.7.164.126:37283)
0808 18:31:37.749693 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:37.749699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:37.750766 (+ 1067us) server_negotiation.cc:407] Connection header received
0808 18:31:37.752111 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:37.752113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:37.752160 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:37.752234 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:37.753290 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.753916 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.755434 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.755662 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.755785 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.757107 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:37.757110 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:37.757112 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:37.757123 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:37.757143 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:37.757150 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:37.757208 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:37.757392 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 18:31:37.757479 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 18:31:37.758639 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.749899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37283 (local address 127.7.164.66:55215)
0808 18:31:37.750611 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:37.750651 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:37.751916 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:37.752398 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:37.752405 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:37.752485 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:37.753131 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:37.753155 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.754052 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.754054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:37.755292 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:37.755305 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.755896 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.755903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:37.756135 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.756802 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:31:37.756806 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:37.756861 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:31:37.756945 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:37.757234 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:37.757246 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:37.757256 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 18:31:37.757385 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 18:31:37.757714 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":118,"threads_started":1}
I20260808 18:31:37.759858 2887 heartbeater.cc:347] Connected to a master server at 127.7.164.126:37283
I20260808 18:31:37.760093 2887 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:37.760603 2887 heartbeater.cc:511] Master 127.7.164.126:37283 requested a full tablet report, sending...
I20260808 18:31:37.761695 2457 ts_manager.cc:194] Registered new tserver with Master: 0a1f2712c5af4bd7bc13c68a8f80df93 (127.7.164.66:41237)
I20260808 18:31:37.762621 2457 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.164.66:55215
W20260808 18:31:37.872570 2891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:37.872818 2891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:37.872856 2891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:37.876828 2891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:37.876883 2891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:37.876935 2891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:31:37.882200 2891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:46061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:37283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:37.883371 2891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:37.884676 2891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:37.894090 2901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:37.894163 2891 server_base.cc:1034] running on GCE node
W20260808 18:31:37.894116 2898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:37.894107 2899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:37.894716 2891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:37.895349 2891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:37.896536 2891 hybrid_clock.cc:648] HybridClock initialized: now 1786213897896520 us; error 26 us; skew 500 ppm
I20260808 18:31:37.898306 2891 webserver.cc:460] Webserver started at http://127.7.164.67:39115/ using document root <none> and password file <none>
I20260808 18:31:37.898861 2891 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:37.898958 2891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:37.899191 2891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:37.901186 2891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "76efaf31f5614630b5c74fe5936e3367"
format_stamp: "Formatted at 2026-08-08 18:31:37 on dist-test-slave-jztv"
server_key: "caf4dd04daa1fb6acaa82b726956172c"
server_key_iv: "8a04dd7d126ee67d6eecbb13c65e99bf"
server_key_version: "encryptionkey@0"
I20260808 18:31:37.901746 2891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "76efaf31f5614630b5c74fe5936e3367"
format_stamp: "Formatted at 2026-08-08 18:31:37 on dist-test-slave-jztv"
server_key: "caf4dd04daa1fb6acaa82b726956172c"
server_key_iv: "8a04dd7d126ee67d6eecbb13c65e99bf"
server_key_version: "encryptionkey@0"
I20260808 18:31:37.905368 2891 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 18:31:37.907752 2907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:37.909042 2891 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:31:37.909199 2891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "76efaf31f5614630b5c74fe5936e3367"
format_stamp: "Formatted at 2026-08-08 18:31:37 on dist-test-slave-jztv"
server_key: "caf4dd04daa1fb6acaa82b726956172c"
server_key_iv: "8a04dd7d126ee67d6eecbb13c65e99bf"
server_key_version: "encryptionkey@0"
I20260808 18:31:37.909317 2891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:37.926085 2891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:37.926831 2891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:37.927050 2891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:37.927677 2891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:37.928758 2891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:37.928869 2891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:37.928967 2891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:37.929008 2891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:37.938298 2891 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:40477
I20260808 18:31:37.938357 3059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:40477 every 8 connection(s)
I20260808 18:31:37.939208 2891 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:31:37.943825 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 2891
I20260808 18:31:37.943954 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:31:37.944234 7825 external_mini_cluster.cc:1428] Setting key e0def72ef08bd140e0820158437c3d06
I20260808 18:31:37.949641 2503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.941136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:59243 (local address 127.7.164.126:37283)
0808 18:31:37.941338 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:37.941369 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:37.942756 (+ 1387us) server_negotiation.cc:407] Connection header received
0808 18:31:37.944541 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:37.944547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:37.944608 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:37.944709 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:37.945929 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.946615 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.947746 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.947981 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.948095 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.949133 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:37.949136 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:37.949139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:37.949152 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:37.949178 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:37.949187 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:37.949263 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:37.949439 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 18:31:37.949500 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 18:31:37.950496 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.941662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37283 (local address 127.7.164.67:59243)
0808 18:31:37.942567 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:37.942626 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:37.944293 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:37.945093 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:37.945104 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:37.945205 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:37.945779 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:37.945794 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.946748 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.946751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:37.947611 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:37.947621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.948135 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.948139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:37.948302 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.948853 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:31:37.948854 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:37.948892 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:31:37.948978 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:37.949277 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:37.949289 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:37.949295 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:31:37.949425 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 18:31:37.949689 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":151,"threads_started":1}
I20260808 18:31:37.951577 3061 heartbeater.cc:347] Connected to a master server at 127.7.164.126:37283
I20260808 18:31:37.951809 3061 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:37.952317 3061 heartbeater.cc:511] Master 127.7.164.126:37283 requested a full tablet report, sending...
I20260808 18:31:37.953327 2457 ts_manager.cc:194] Registered new tserver with Master: 76efaf31f5614630b5c74fe5936e3367 (127.7.164.67:40477)
I20260808 18:31:37.953898 2457 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.164.67:59243
I20260808 18:31:37.958069 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:31:37.963956 2503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.960208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42538 (local address 127.7.164.126:37283)
0808 18:31:37.960357 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:37.960361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:37.960454 (+ 93us) server_negotiation.cc:407] Connection header received
0808 18:31:37.960560 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:37.960562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:37.960603 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:37.960670 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:37.960939 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.961441 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.962336 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.962545 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.962620 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.963471 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:37.963474 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:37.963476 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:37.963486 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:37.963501 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:37.963508 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:37.963598 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:37.963728 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 18:31:37.963789 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:31:37.966737 2457 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:31:37.969604 2457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:31:37.984352 3079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.978702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:38395 (local address 127.0.0.1:59958)
0808 18:31:37.979618 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:37.979630 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:37.979717 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:37.980155 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:37.980159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:37.980177 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:37.980270 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:37.980278 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.981415 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.981419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:37.982859 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:37.982869 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.983762 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.983766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:37.983872 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.983916 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 18:31:37.984042 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 18:31:37.984167 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":5,"thread_start_us":90,"threads_started":1}
I20260808 18:31:37.984359 3080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.979102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59958 (local address 127.7.164.65:38395)
0808 18:31:37.979584 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:37.979595 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:37.979722 (+ 127us) server_negotiation.cc:407] Connection header received
0808 18:31:37.979810 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:37.979818 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:37.980005 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:37.980150 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:37.980397 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.981280 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.982993 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.983628 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.983737 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.983937 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:37.984071 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 18:31:37.984173 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":155,"threads_started":1}
I20260808 18:31:37.985009 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.977950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:40477 (local address 127.0.0.1:51262)
0808 18:31:37.978821 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:37.978871 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:37.979029 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:37.979867 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:37.979874 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:37.979898 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:37.980071 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:37.980079 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.981604 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.981609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:37.983293 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:37.983305 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.984568 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.984572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:37.984671 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.984689 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 18:31:37.984790 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 18:31:37.984858 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":211,"thread_start_us":88,"threads_started":1}
I20260808 18:31:37.985105 3075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.978287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.7.164.66:41237)
0808 18:31:37.978956 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:37.978968 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:37.979274 (+ 306us) server_negotiation.cc:407] Connection header received
0808 18:31:37.979529 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:37.979535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:37.979751 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:37.979903 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:37.980215 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.981461 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.983449 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.984191 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.984290 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.984466 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:37.984804 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 18:31:37.984908 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":120,"threads_started":1}
I20260808 18:31:37.985450 3073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.978000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:41237 (local address 127.0.0.1:59894)
0808 18:31:37.979186 (+ 1186us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:37.979199 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:37.979356 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:37.979887 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:37.979889 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:37.979912 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:37.980071 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:37.980079 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.981648 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.981650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:37.983294 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:37.983305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.984543 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.984548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:37.984639 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.984668 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 18:31:37.985245 (+ 577us) client_negotiation.cc:240] Negotiation successful
0808 18:31:37.985324 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1118,"mutex_wait_us":123,"spinlock_wait_cycles":2432,"thread_start_us":86,"threads_started":1}
I20260808 18:31:37.985936 3076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:37.978335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51262 (local address 127.7.164.67:40477)
0808 18:31:37.978753 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:37.978765 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:37.979029 (+ 264us) server_negotiation.cc:407] Connection header received
0808 18:31:37.979243 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:37.979252 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:37.979489 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:37.979644 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:37.980214 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.981461 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.983438 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:37.984194 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:37.984298 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:37.984496 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:37.985615 (+ 1119us) server_negotiation.cc:299] Negotiation successful
0808 18:31:37.985738 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":142,"threads_started":1}
I20260808 18:31:37.988090 2796 tablet_service.cc:1467] Processing CreateTablet for tablet f1a0afe4934f493dab9619401adf3b0d (DEFAULT_TABLE table=test-table [id=b02b2b0f3ab443dfbb026c22c66a607d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:37.988107 2619 tablet_service.cc:1467] Processing CreateTablet for tablet f1a0afe4934f493dab9619401adf3b0d (DEFAULT_TABLE table=test-table [id=b02b2b0f3ab443dfbb026c22c66a607d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:37.988478 2971 tablet_service.cc:1467] Processing CreateTablet for tablet f1a0afe4934f493dab9619401adf3b0d (DEFAULT_TABLE table=test-table [id=b02b2b0f3ab443dfbb026c22c66a607d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:37.989204 2796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1a0afe4934f493dab9619401adf3b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:37.989205 2619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1a0afe4934f493dab9619401adf3b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:37.989389 2971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1a0afe4934f493dab9619401adf3b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:37.995430 3081 tablet_bootstrap.cc:492] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93: Bootstrap starting.
I20260808 18:31:37.996981 3083 tablet_bootstrap.cc:492] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318: Bootstrap starting.
I20260808 18:31:37.996981 3082 tablet_bootstrap.cc:492] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367: Bootstrap starting.
I20260808 18:31:37.997572 3081 tablet_bootstrap.cc:654] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:37.998399 3081 log.cc:824] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:37.999127 3082 tablet_bootstrap.cc:654] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:37.999127 3083 tablet_bootstrap.cc:654] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:38.000011 3082 log.cc:824] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:38.000001 3083 log.cc:824] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:38.000741 3081 tablet_bootstrap.cc:492] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93: No bootstrap required, opened a new log
I20260808 18:31:38.000965 3081 ts_tablet_manager.cc:1397] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:31:38.002095 3082 tablet_bootstrap.cc:492] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367: No bootstrap required, opened a new log
I20260808 18:31:38.002319 3082 ts_tablet_manager.cc:1397] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:31:38.002344 3083 tablet_bootstrap.cc:492] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318: No bootstrap required, opened a new log
I20260808 18:31:38.002525 3083 ts_tablet_manager.cc:1397] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:31:38.004313 3081 raft_consensus.cc:348] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76efaf31f5614630b5c74fe5936e3367" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40477 } } peers { permanent_uuid: "0a1f2712c5af4bd7bc13c68a8f80df93" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41237 } } peers { permanent_uuid: "6ad9268bd0134d10aef05979c25ac318" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38395 } }
I20260808 18:31:38.004643 3081 raft_consensus.cc:374] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:38.004724 3081 raft_consensus.cc:729] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1f2712c5af4bd7bc13c68a8f80df93, State: Initialized, Role: FOLLOWER
I20260808 18:31:38.005184 3081 consensus_queue.cc:260] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76efaf31f5614630b5c74fe5936e3367" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40477 } } peers { permanent_uuid: "0a1f2712c5af4bd7bc13c68a8f80df93" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41237 } } peers { permanent_uuid: "6ad9268bd0134d10aef05979c25ac318" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38395 } }
I20260808 18:31:38.005820 2887 heartbeater.cc:503] Master 127.7.164.126:37283 was elected leader, sending a full tablet report...
I20260808 18:31:38.006676 3082 raft_consensus.cc:348] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76efaf31f5614630b5c74fe5936e3367" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40477 } } peers { permanent_uuid: "0a1f2712c5af4bd7bc13c68a8f80df93" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41237 } } peers { permanent_uuid: "6ad9268bd0134d10aef05979c25ac318" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38395 } }
I20260808 18:31:38.006676 3083 raft_consensus.cc:348] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76efaf31f5614630b5c74fe5936e3367" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40477 } } peers { permanent_uuid: "0a1f2712c5af4bd7bc13c68a8f80df93" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41237 } } peers { permanent_uuid: "6ad9268bd0134d10aef05979c25ac318" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38395 } }
I20260808 18:31:38.006927 3083 raft_consensus.cc:374] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:38.007016 3082 raft_consensus.cc:374] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:38.007023 3083 raft_consensus.cc:729] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9268bd0134d10aef05979c25ac318, State: Initialized, Role: FOLLOWER
I20260808 18:31:38.007125 3082 raft_consensus.cc:729] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76efaf31f5614630b5c74fe5936e3367, State: Initialized, Role: FOLLOWER
I20260808 18:31:38.007521 3083 consensus_queue.cc:260] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76efaf31f5614630b5c74fe5936e3367" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40477 } } peers { permanent_uuid: "0a1f2712c5af4bd7bc13c68a8f80df93" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41237 } } peers { permanent_uuid: "6ad9268bd0134d10aef05979c25ac318" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38395 } }
I20260808 18:31:38.007512 3082 consensus_queue.cc:260] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76efaf31f5614630b5c74fe5936e3367" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40477 } } peers { permanent_uuid: "0a1f2712c5af4bd7bc13c68a8f80df93" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41237 } } peers { permanent_uuid: "6ad9268bd0134d10aef05979c25ac318" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38395 } }
I20260808 18:31:38.008224 2706 heartbeater.cc:503] Master 127.7.164.126:37283 was elected leader, sending a full tablet report...
I20260808 18:31:38.008550 3082 ts_tablet_manager.cc:1428] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:31:38.008870 3061 heartbeater.cc:503] Master 127.7.164.126:37283 was elected leader, sending a full tablet report...
I20260808 18:31:38.009611 3083 ts_tablet_manager.cc:1428] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 18:31:38.010143 3081 ts_tablet_manager.cc:1428] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260808 18:31:38.030110 3089 raft_consensus.cc:482] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:31:38.030383 3089 raft_consensus.cc:504] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76efaf31f5614630b5c74fe5936e3367" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40477 } } peers { permanent_uuid: "0a1f2712c5af4bd7bc13c68a8f80df93" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41237 } } peers { permanent_uuid: "6ad9268bd0134d10aef05979c25ac318" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38395 } }
I20260808 18:31:38.031661 3089 leader_election.cc:288] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a1f2712c5af4bd7bc13c68a8f80df93 (127.7.164.66:41237), 6ad9268bd0134d10aef05979c25ac318 (127.7.164.65:38395)
I20260808 18:31:38.035326 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:38.031846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:41237 (local address 127.7.164.67:56395)
0808 18:31:38.031986 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:38.032003 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:38.032101 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:38.032465 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:38.032467 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:38.032487 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:38.032616 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:38.032622 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:38.033371 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:38.033378 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:38.034179 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:38.034188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:38.034867 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:38.034871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:38.034987 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:38.035011 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:31:38.035106 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 18:31:38.035181 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 18:31:38.035324 3075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:38.032087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:56395 (local address 127.7.164.66:41237)
0808 18:31:38.032219 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:38.032225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:38.032241 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:31:38.032297 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:38.032300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:38.032358 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:38.032452 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:38.032740 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:38.033254 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:38.034340 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:38.034736 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:38.034875 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:38.034928 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:38.035123 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 18:31:38.035197 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:31:38.035795 3080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:38.032171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:43753 (local address 127.7.164.65:38395)
0808 18:31:38.032295 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:38.032300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:38.032537 (+ 237us) server_negotiation.cc:407] Connection header received
0808 18:31:38.032693 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:38.032696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:38.032748 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:38.032846 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:38.033126 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:38.033615 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:38.034752 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:38.035156 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:38.035296 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:38.035346 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:38.035572 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 18:31:38.035662 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:31:38.035808 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:38.031962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:38395 (local address 127.7.164.67:43753)
0808 18:31:38.032424 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:38.032436 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:38.032571 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:38.032860 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:38.032863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:38.032880 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:38.033005 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:38.033010 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:38.033737 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:38.033742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:38.034620 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:38.034630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:38.035299 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:38.035303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:38.035434 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:38.035469 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 18:31:38.035544 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 18:31:38.035611 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":143,"threads_started":1}
I20260808 18:31:38.036121 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a0afe4934f493dab9619401adf3b0d" candidate_uuid: "76efaf31f5614630b5c74fe5936e3367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1f2712c5af4bd7bc13c68a8f80df93" is_pre_election: true
I20260808 18:31:38.036465 2828 raft_consensus.cc:2454] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76efaf31f5614630b5c74fe5936e3367 in term 0.
I20260808 18:31:38.036616 2642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a0afe4934f493dab9619401adf3b0d" candidate_uuid: "76efaf31f5614630b5c74fe5936e3367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9268bd0134d10aef05979c25ac318" is_pre_election: true
I20260808 18:31:38.036943 2642 raft_consensus.cc:2454] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76efaf31f5614630b5c74fe5936e3367 in term 0.
I20260808 18:31:38.037066 2910 leader_election.cc:302] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1f2712c5af4bd7bc13c68a8f80df93, 76efaf31f5614630b5c74fe5936e3367; no voters:
I20260808 18:31:38.037370 3089 raft_consensus.cc:2781] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:31:38.037463 3089 raft_consensus.cc:482] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:31:38.037523 3089 raft_consensus.cc:3037] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:38.038626 3089 raft_consensus.cc:504] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76efaf31f5614630b5c74fe5936e3367" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40477 } } peers { permanent_uuid: "0a1f2712c5af4bd7bc13c68a8f80df93" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41237 } } peers { permanent_uuid: "6ad9268bd0134d10aef05979c25ac318" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38395 } }
I20260808 18:31:38.039047 3089 leader_election.cc:288] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [CANDIDATE]: Term 1 election: Requested vote from peers 0a1f2712c5af4bd7bc13c68a8f80df93 (127.7.164.66:41237), 6ad9268bd0134d10aef05979c25ac318 (127.7.164.65:38395)
I20260808 18:31:38.039440 2642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a0afe4934f493dab9619401adf3b0d" candidate_uuid: "76efaf31f5614630b5c74fe5936e3367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9268bd0134d10aef05979c25ac318"
I20260808 18:31:38.039556 2642 raft_consensus.cc:3037] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:38.040525 2642 raft_consensus.cc:2454] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76efaf31f5614630b5c74fe5936e3367 in term 1.
I20260808 18:31:38.040757 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1a0afe4934f493dab9619401adf3b0d" candidate_uuid: "76efaf31f5614630b5c74fe5936e3367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1f2712c5af4bd7bc13c68a8f80df93"
I20260808 18:31:38.040840 2911 leader_election.cc:302] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9268bd0134d10aef05979c25ac318, 76efaf31f5614630b5c74fe5936e3367; no voters:
I20260808 18:31:38.040913 2828 raft_consensus.cc:3037] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:38.041056 3089 raft_consensus.cc:2781] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:38.041360 3089 raft_consensus.cc:686] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [term 1 LEADER]: Becoming Leader. State: Replica: 76efaf31f5614630b5c74fe5936e3367, State: Running, Role: LEADER
I20260808 18:31:38.041718 3089 consensus_queue.cc:237] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76efaf31f5614630b5c74fe5936e3367" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40477 } } peers { permanent_uuid: "0a1f2712c5af4bd7bc13c68a8f80df93" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41237 } } peers { permanent_uuid: "6ad9268bd0134d10aef05979c25ac318" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38395 } }
I20260808 18:31:38.042282 2828 raft_consensus.cc:2454] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76efaf31f5614630b5c74fe5936e3367 in term 1.
I20260808 18:31:38.045384 2457 catalog_manager.cc:5495] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76efaf31f5614630b5c74fe5936e3367 (127.7.164.67). New cstate: current_term: 1 leader_uuid: "76efaf31f5614630b5c74fe5936e3367" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76efaf31f5614630b5c74fe5936e3367" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 40477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1f2712c5af4bd7bc13c68a8f80df93" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9268bd0134d10aef05979c25ac318" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38395 } health_report { overall_health: UNKNOWN } } }
W20260808 18:31:38.058992 2707 tablet.cc:2369] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:31:38.064920 3076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:38.060462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51270 (local address 127.7.164.67:40477)
0808 18:31:38.060622 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:38.060626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:38.060640 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:31:38.060709 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:38.060711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:38.060765 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:38.060853 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:31:38.061096 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:38.061873 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:38.063146 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:38.063464 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:38.063570 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:38.063916 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:31:38.064424 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:31:38.064567 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:38.064724 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 18:31:38.064762 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 18:31:38.074777 2828 raft_consensus.cc:1261] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93 [term 1 FOLLOWER]: Refusing update from remote peer 76efaf31f5614630b5c74fe5936e3367: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:31:38.074779 2642 raft_consensus.cc:1261] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318 [term 1 FOLLOWER]: Refusing update from remote peer 76efaf31f5614630b5c74fe5936e3367: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:31:38.075515 3091 consensus_queue.cc:1035] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9268bd0134d10aef05979c25ac318" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:31:38.075702 3089 consensus_queue.cc:1035] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1f2712c5af4bd7bc13c68a8f80df93" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 41237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:31:38.083923 3096 mvcc.cc:205] Tried to move back new op lower bound from 7316332126507171840 to 7316332126382845952. Current Snapshot: MvccSnapshot[applied={T|T < 7316332126507171840}]
I20260808 18:31:38.084342 3095 mvcc.cc:205] Tried to move back new op lower bound from 7316332126507171840 to 7316332126382845952. Current Snapshot: MvccSnapshot[applied={T|T < 7316332126507171840}]
I20260808 18:31:38.084559 3097 mvcc.cc:205] Tried to move back new op lower bound from 7316332126507171840 to 7316332126382845952. Current Snapshot: MvccSnapshot[applied={T|T < 7316332126507171840}]
I20260808 18:31:38.093416 2456 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:31:38.093562 2456 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:31:38.096037 2456 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a861606983154ebd8e74efd128399f6e: Sending DeleteTablet for 3 replicas of tablet f1a0afe4934f493dab9619401adf3b0d
I20260808 18:31:38.096884 2619 tablet_service.cc:1514] Processing DeleteTablet for tablet f1a0afe4934f493dab9619401adf3b0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:38 UTC) from {username='slave'} at 127.0.0.1:59958
I20260808 18:31:38.096978 2796 tablet_service.cc:1514] Processing DeleteTablet for tablet f1a0afe4934f493dab9619401adf3b0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:38 UTC) from {username='slave'} at 127.0.0.1:59894
I20260808 18:31:38.097047 2971 tablet_service.cc:1514] Processing DeleteTablet for tablet f1a0afe4934f493dab9619401adf3b0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:38 UTC) from {username='slave'} at 127.0.0.1:51262
I20260808 18:31:38.097678 3106 tablet_replica.cc:330] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93: stopping tablet replica
I20260808 18:31:38.097754 3107 tablet_replica.cc:330] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318: stopping tablet replica
I20260808 18:31:38.098104 3107 raft_consensus.cc:2229] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:31:38.098104 3106 raft_consensus.cc:2229] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:31:38.098161 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 2539
I20260808 18:31:38.098460 3107 raft_consensus.cc:2258] T f1a0afe4934f493dab9619401adf3b0d P 6ad9268bd0134d10aef05979c25ac318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:31:38.098803 3108 tablet_replica.cc:330] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367: stopping tablet replica
I20260808 18:31:38.099172 3108 raft_consensus.cc:2229] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:31:38.099568 3106 raft_consensus.cc:2258] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:31:38.099591 3108 raft_consensus.cc:2258] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:31:38.102742 3106 ts_tablet_manager.cc:1905] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:31:38.105790 3106 ts_tablet_manager.cc:1918] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:31:38.105938 3106 log.cc:1196] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/f1a0afe4934f493dab9619401adf3b0d
I20260808 18:31:38.106343 3106 ts_tablet_manager.cc:1939] T f1a0afe4934f493dab9619401adf3b0d P 0a1f2712c5af4bd7bc13c68a8f80df93: Deleting consensus metadata
I20260808 18:31:38.108106 2436 catalog_manager.cc:4841] TS 0a1f2712c5af4bd7bc13c68a8f80df93 (127.7.164.66:41237): tablet f1a0afe4934f493dab9619401adf3b0d (table test-table [id=b02b2b0f3ab443dfbb026c22c66a607d]) successfully deleted
I20260808 18:31:38.108709 3108 ts_tablet_manager.cc:1905] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:31:38.112262 3108 ts_tablet_manager.cc:1918] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:31:38.112424 3108 log.cc:1196] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/f1a0afe4934f493dab9619401adf3b0d
I20260808 18:31:38.112753 3108 ts_tablet_manager.cc:1939] T f1a0afe4934f493dab9619401adf3b0d P 76efaf31f5614630b5c74fe5936e3367: Deleting consensus metadata
I20260808 18:31:38.114248 2437 catalog_manager.cc:4841] TS 76efaf31f5614630b5c74fe5936e3367 (127.7.164.67:40477): tablet f1a0afe4934f493dab9619401adf3b0d (table test-table [id=b02b2b0f3ab443dfbb026c22c66a607d]) successfully deleted
W20260808 18:31:38.114562 2437 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:38395 (error 108)
I20260808 18:31:38.114603 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 2709
I20260808 18:31:38.119671 3079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:38.119364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:38395 (local address 127.0.0.1:59974)
0808 18:31:38.119514 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:38.119589 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:38395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 18:31:38.119905 2437 catalog_manager.cc:4584] TS 6ad9268bd0134d10aef05979c25ac318 (127.7.164.65:38395): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f1a0afe4934f493dab9619401adf3b0d: Network error: Client connection negotiation failed: client connection to 127.7.164.65:38395: connect: Connection refused (error 111)
I20260808 18:31:38.128945 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 2891
I20260808 18:31:38.140959 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 2413
2026-08-08T18:31:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1048 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:38 dist-test-slave-jztv krb5kdc[3114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:38 dist-test-slave-jztv krb5kdc[3114](info): set up 2 sockets
Aug 08 18:31:38 dist-test-slave-jztv krb5kdc[3114](info): commencing operation
krb5kdc: starting...
W20260808 18:31:39.395347 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:31:39 dist-test-slave-jztv krb5kdc[3114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:31:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:31:39.526835 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:45927
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:45927
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:39.642132 3134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:39.642488 3134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:39.642530 3134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:39.646579 3134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:39.646634 3134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:39.646653 3134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:39.646670 3134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:39.646718 3134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:39.652639 3134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:45927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:45927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:39.653760 3134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:39.654843 3134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:31:39.663662 3134 server_base.cc:1034] running on GCE node
W20260808 18:31:39.663633 3143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:39.663622 3142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:39.663637 3145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:39.664531 3134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:39.665548 3134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:39.666723 3134 hybrid_clock.cc:648] HybridClock initialized: now 1786213899666690 us; error 42 us; skew 500 ppm
Aug 08 18:31:39 dist-test-slave-jztv krb5kdc[3114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213899, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:39.672649 3134 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:31:39.673661 3134 webserver.cc:460] Webserver started at http://127.7.164.126:34895/ using document root <none> and password file <none>
I20260808 18:31:39.674289 3134 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:39.674360 3134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:39.674544 3134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:39.676549 3134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "adb95dadff1d47199fe166ceb7e1cd16"
format_stamp: "Formatted at 2026-08-08 18:31:39 on dist-test-slave-jztv"
server_key: "0a510ccb7c176abef269dbab63216214"
server_key_iv: "875319ab566c2cf2bd70674cae7ed6e1"
server_key_version: "encryptionkey@0"
I20260808 18:31:39.677073 3134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "adb95dadff1d47199fe166ceb7e1cd16"
format_stamp: "Formatted at 2026-08-08 18:31:39 on dist-test-slave-jztv"
server_key: "0a510ccb7c176abef269dbab63216214"
server_key_iv: "875319ab566c2cf2bd70674cae7ed6e1"
server_key_version: "encryptionkey@0"
I20260808 18:31:39.680852 3134 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:31:39.683408 3158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:39.684664 3134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:31:39.684808 3134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "adb95dadff1d47199fe166ceb7e1cd16"
format_stamp: "Formatted at 2026-08-08 18:31:39 on dist-test-slave-jztv"
server_key: "0a510ccb7c176abef269dbab63216214"
server_key_iv: "875319ab566c2cf2bd70674cae7ed6e1"
server_key_version: "encryptionkey@0"
I20260808 18:31:39.684921 3134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:39.712280 3134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:39.715451 3134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:39.715656 3134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:39.722985 3134 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:45927
I20260808 18:31:39.723002 3235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:45927 every 8 connection(s)
I20260808 18:31:39.723908 3134 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:39.724398 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 3134
I20260808 18:31:39.724555 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:39.724808 7825 external_mini_cluster.cc:1428] Setting key 207b26e1563d4094d843f181490b483e
I20260808 18:31:39.727126 3236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:39.734068 3236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16: Bootstrap starting.
Aug 08 18:31:39 dist-test-slave-jztv krb5kdc[3114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:31:39.736914 3236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:39.737794 3236 log.cc:824] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:39.740183 3236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16: No bootstrap required, opened a new log
I20260808 18:31:39.740254 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:39.726169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57214 (local address 127.7.164.126:45927)
0808 18:31:39.726652 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:39.726672 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:39.726714 (+ 42us) server_negotiation.cc:407] Connection header received
0808 18:31:39.727583 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:39.727606 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:39.728006 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:39.728362 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:39.729056 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:39.730633 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:39.731508 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:39.732192 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:39.732351 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:39.735026 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:39.735033 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:39.735049 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:39.735082 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:39.737091 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:39.737557 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:39.737563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:39.737569 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:39.737724 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:39.738056 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:39.738060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:39.738061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:39.738545 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:39.738736 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:39.739031 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 18:31:39.739322 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":127,"threads_started":1}
I20260808 18:31:39.744053 3236 raft_consensus.cc:348] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb95dadff1d47199fe166ceb7e1cd16" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45927 } }
I20260808 18:31:39.744273 3236 raft_consensus.cc:374] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:39.744319 3236 raft_consensus.cc:729] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adb95dadff1d47199fe166ceb7e1cd16, State: Initialized, Role: FOLLOWER
I20260808 18:31:39.744674 3236 consensus_queue.cc:260] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb95dadff1d47199fe166ceb7e1cd16" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45927 } }
I20260808 18:31:39.744786 3236 raft_consensus.cc:388] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:39.744825 3236 raft_consensus.cc:482] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:39.744887 3236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:39.745769 3236 raft_consensus.cc:504] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb95dadff1d47199fe166ceb7e1cd16" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45927 } }
I20260808 18:31:39.746184 3236 leader_election.cc:302] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adb95dadff1d47199fe166ceb7e1cd16; no voters:
I20260808 18:31:39.746641 3236 leader_election.cc:288] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:39.746771 3244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:39.747045 3244 raft_consensus.cc:686] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [term 1 LEADER]: Becoming Leader. State: Replica: adb95dadff1d47199fe166ceb7e1cd16, State: Running, Role: LEADER
I20260808 18:31:39.747422 3244 consensus_queue.cc:237] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb95dadff1d47199fe166ceb7e1cd16" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45927 } }
I20260808 18:31:39.747900 3236 sys_catalog.cc:564] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:39.749105 3246 sys_catalog.cc:455] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adb95dadff1d47199fe166ceb7e1cd16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb95dadff1d47199fe166ceb7e1cd16" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45927 } } }
I20260808 18:31:39.749251 3246 sys_catalog.cc:458] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:39.749555 3247 sys_catalog.cc:455] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adb95dadff1d47199fe166ceb7e1cd16. Latest consensus state: current_term: 1 leader_uuid: "adb95dadff1d47199fe166ceb7e1cd16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb95dadff1d47199fe166ceb7e1cd16" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 45927 } } }
I20260808 18:31:39.749643 3255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:39.750418 3247 sys_catalog.cc:458] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:39.753086 3255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:39.759779 3255 catalog_manager.cc:1347] Generated new cluster ID: 9cf8bb9df4754e1db0597929957ebcb7
I20260808 18:31:39.759886 3255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:39.767094 3255 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:39.768065 3255 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:39.776508 3255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16: Generated new TSK 0
I20260808 18:31:39.777232 3255 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.164.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
Entry for principal kudu/127.7.164.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab.
I20260808 18:31:39.814254 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45927
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:39.933033 3269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:39.933290 3269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:39.933333 3269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:39.937572 3269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:39.937636 3269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:39.937736 3269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:31:39.943018 3269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.65.keytab
--principal=kudu/127.7.164.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:39.944100 3269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:39.945163 3269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:39.956128 3275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:39.956171 3276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:39.956140 3279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:39.956328 3269 server_base.cc:1034] running on GCE node
I20260808 18:31:39.956885 3269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:39.957594 3269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:39.958794 3269 hybrid_clock.cc:648] HybridClock initialized: now 1786213899958780 us; error 36 us; skew 500 ppm
Aug 08 18:31:39 dist-test-slave-jztv krb5kdc[3114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213899, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:39.961575 3269 init.cc:377] Logged in from keytab as kudu/127.7.164.65@KRBTEST.COM (short username kudu)
I20260808 18:31:39.962558 3269 webserver.cc:460] Webserver started at http://127.7.164.65:35261/ using document root <none> and password file <none>
I20260808 18:31:39.963155 3269 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:39.963253 3269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:39.963496 3269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:39.965580 3269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "def4da0e5a414521a5bedad8218611ae"
format_stamp: "Formatted at 2026-08-08 18:31:39 on dist-test-slave-jztv"
server_key: "124cb35a6a4e1c7d0fe2daaf03140bf0"
server_key_iv: "8b9d9706a10a11240f0a00ddfcbc9a81"
server_key_version: "encryptionkey@0"
I20260808 18:31:39.966277 3269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "def4da0e5a414521a5bedad8218611ae"
format_stamp: "Formatted at 2026-08-08 18:31:39 on dist-test-slave-jztv"
server_key: "124cb35a6a4e1c7d0fe2daaf03140bf0"
server_key_iv: "8b9d9706a10a11240f0a00ddfcbc9a81"
server_key_version: "encryptionkey@0"
I20260808 18:31:39.970556 3269 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 18:31:39.973215 3292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:39.974702 3269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:31:39.974859 3269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "def4da0e5a414521a5bedad8218611ae"
format_stamp: "Formatted at 2026-08-08 18:31:39 on dist-test-slave-jztv"
server_key: "124cb35a6a4e1c7d0fe2daaf03140bf0"
server_key_iv: "8b9d9706a10a11240f0a00ddfcbc9a81"
server_key_version: "encryptionkey@0"
I20260808 18:31:39.974987 3269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:40.035606 3269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:40.039304 3269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:40.039572 3269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:40.040222 3269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:40.041115 3269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:40.041191 3269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:40.041263 3269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:40.041316 3269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:40.050776 3269 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:43937
I20260808 18:31:40.050803 3450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:43937 every 8 connection(s)
I20260808 18:31:40.051712 3269 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:31:40.053089 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 3269
I20260808 18:31:40.053222 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:31:40.053552 7825 external_mini_cluster.cc:1428] Setting key 386699704064365725c8f085293e21da
Aug 08 18:31:40 dist-test-slave-jztv krb5kdc[3114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213899, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.65@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.164.66@KRBTEST.COM; defaulting to no policy
I20260808 18:31:40.067662 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.053745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:45629 (local address 127.7.164.126:45927)
0808 18:31:40.053920 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:40.053927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:40.054894 (+ 967us) server_negotiation.cc:407] Connection header received
0808 18:31:40.056906 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:40.056909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:40.056955 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:40.057028 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:40.058504 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.059367 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.060505 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.060943 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.061071 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.063865 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:40.063869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:40.063872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:40.063904 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:40.065555 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:40.066062 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:40.066068 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:40.066070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:40.066160 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:40.066586 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:40.066589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:40.066590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:40.066791 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:40.066869 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:40.067347 (+ 478us) server_negotiation.cc:299] Negotiation successful
0808 18:31:40.067458 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.66@KRBTEST.COM" created.
I20260808 18:31:40.068508 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.054250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45927 (local address 127.7.164.65:45629)
0808 18:31:40.054751 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:40.054793 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:40.056667 (+ 1874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:40.057245 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:40.057257 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:40.057768 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:40.058333 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.058350 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.059507 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.059509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.060371 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.060379 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.061108 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.061111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.061253 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.061902 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:31:40.061918 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:40.063681 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:40.065664 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:40.065669 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:40.065681 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:40.065934 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:40.066299 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:40.066302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:40.066303 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:40.066491 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:40.066883 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:40.066888 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:40.067126 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 18:31:40.067274 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 18:31:40.067523 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":167,"threads_started":1}
I20260808 18:31:40.069883 3452 heartbeater.cc:347] Connected to a master server at 127.7.164.126:45927
I20260808 18:31:40.070152 3452 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:40.070670 3452 heartbeater.cc:511] Master 127.7.164.126:45927 requested a full tablet report, sending...
I20260808 18:31:40.072388 3185 ts_manager.cc:194] Registered new tserver with Master: def4da0e5a414521a5bedad8218611ae (127.7.164.65:43937)
I20260808 18:31:40.074098 3185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.65@KRBTEST.COM'} at 127.7.164.65:45629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
Entry for principal kudu/127.7.164.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab.
I20260808 18:31:40.085425 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45927
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:40.198009 3458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:40.198293 3458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:40.198397 3458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:40.202487 3458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:40.202543 3458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:40.202646 3458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:31:40.207820 3458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.66.keytab
--principal=kudu/127.7.164.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:40.208870 3458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:40.209892 3458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:40.220176 3469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:40.220196 3466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:40.220213 3464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:40.220232 3458 server_base.cc:1034] running on GCE node
I20260808 18:31:40.220865 3458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:40.221500 3458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:40.222689 3458 hybrid_clock.cc:648] HybridClock initialized: now 1786213900222648 us; error 67 us; skew 500 ppm
Aug 08 18:31:40 dist-test-slave-jztv krb5kdc[3114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213900, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:40.225430 3458 init.cc:377] Logged in from keytab as kudu/127.7.164.66@KRBTEST.COM (short username kudu)
I20260808 18:31:40.226439 3458 webserver.cc:460] Webserver started at http://127.7.164.66:43739/ using document root <none> and password file <none>
I20260808 18:31:40.227025 3458 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:40.227123 3458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:40.227368 3458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:40.229400 3458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "eceb9e9fad444679aa883899ad40ec47"
format_stamp: "Formatted at 2026-08-08 18:31:40 on dist-test-slave-jztv"
server_key: "adafadc612b149de7a47316251118fd0"
server_key_iv: "84c1b82e44522de8cfe7ff93aa88cf5b"
server_key_version: "encryptionkey@0"
I20260808 18:31:40.229992 3458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "eceb9e9fad444679aa883899ad40ec47"
format_stamp: "Formatted at 2026-08-08 18:31:40 on dist-test-slave-jztv"
server_key: "adafadc612b149de7a47316251118fd0"
server_key_iv: "84c1b82e44522de8cfe7ff93aa88cf5b"
server_key_version: "encryptionkey@0"
I20260808 18:31:40.233855 3458 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 18:31:40.236645 3477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:40.237927 3458 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:31:40.238082 3458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "eceb9e9fad444679aa883899ad40ec47"
format_stamp: "Formatted at 2026-08-08 18:31:40 on dist-test-slave-jztv"
server_key: "adafadc612b149de7a47316251118fd0"
server_key_iv: "84c1b82e44522de8cfe7ff93aa88cf5b"
server_key_version: "encryptionkey@0"
I20260808 18:31:40.238206 3458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:40.259600 3458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:40.262840 3458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:40.263067 3458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:40.263685 3458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:40.264549 3458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:40.264637 3458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:40.264732 3458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:40.264786 3458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:40.274211 3458 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:44993
I20260808 18:31:40.274269 3656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:44993 every 8 connection(s)
I20260808 18:31:40.275179 3458 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:31:40.282824 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 3458
I20260808 18:31:40.282956 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:31:40.283324 7825 external_mini_cluster.cc:1428] Setting key 878587ec389b63f4506d1b487b3ba5fa
Aug 08 18:31:40 dist-test-slave-jztv krb5kdc[3114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213900, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.66@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:31:40.293149 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.277134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:46767 (local address 127.7.164.126:45927)
0808 18:31:40.277273 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:40.277276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:40.278067 (+ 791us) server_negotiation.cc:407] Connection header received
0808 18:31:40.279994 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:40.280001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:40.280067 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:40.280177 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:40.282119 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.283046 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.284819 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.285286 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.285399 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.289192 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:40.289199 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:40.289205 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:40.289239 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:40.290993 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:40.291542 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:40.291547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:40.291549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:40.291633 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:40.292014 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:40.292018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:40.292020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:40.292211 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:40.292324 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:40.292850 (+ 526us) server_negotiation.cc:299] Negotiation successful
0808 18:31:40.292981 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:31:40.293978 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.277453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45927 (local address 127.7.164.66:46767)
0808 18:31:40.277908 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:40.277950 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:40.279704 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:40.280428 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:40.280437 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:40.281062 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:40.281888 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.281925 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.283252 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.283257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.284614 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.284625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.285478 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.285483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.285712 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.287050 (+ 1338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:31:40.287077 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:40.288912 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:40.291116 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:40.291122 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:40.291139 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:40.291412 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:40.291734 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:40.291738 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:40.291740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:40.291891 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:40.292324 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:40.292332 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:40.292596 (+ 264us) client_negotiation.cc:769] Sending connection context
0808 18:31:40.292793 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 18:31:40.293140 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":120,"threads_started":1}
WARNING: no policy specified for kudu/127.7.164.67@KRBTEST.COM; defaulting to no policy
I20260808 18:31:40.295341 3657 heartbeater.cc:347] Connected to a master server at 127.7.164.126:45927
I20260808 18:31:40.295593 3657 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:40.296162 3657 heartbeater.cc:511] Master 127.7.164.126:45927 requested a full tablet report, sending...
I20260808 18:31:40.297320 3185 ts_manager.cc:194] Registered new tserver with Master: eceb9e9fad444679aa883899ad40ec47 (127.7.164.66:44993)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.67@KRBTEST.COM" created.
I20260808 18:31:40.298117 3185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.66@KRBTEST.COM'} at 127.7.164.66:46767
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
Entry for principal kudu/127.7.164.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab.
I20260808 18:31:40.313874 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45927
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:40.430372 3667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:40.430639 3667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:40.430735 3667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:40.434820 3667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:40.434880 3667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:40.434940 3667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:31:40.440402 3667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.67.keytab
--principal=kudu/127.7.164.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:45927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:40.441587 3667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:40.442741 3667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:31:40.452873 3667 server_base.cc:1034] running on GCE node
W20260808 18:31:40.452816 3676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:40.452776 3674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:40.452793 3672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:40.453609 3667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:40.454346 3667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:40.455556 3667 hybrid_clock.cc:648] HybridClock initialized: now 1786213900455550 us; error 41 us; skew 500 ppm
Aug 08 18:31:40 dist-test-slave-jztv krb5kdc[3114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213900, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:40.458348 3667 init.cc:377] Logged in from keytab as kudu/127.7.164.67@KRBTEST.COM (short username kudu)
I20260808 18:31:40.459189 3667 webserver.cc:460] Webserver started at http://127.7.164.67:44803/ using document root <none> and password file <none>
I20260808 18:31:40.459761 3667 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:40.459856 3667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:40.460079 3667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:40.462203 3667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "7a94b818eb7843ca9e99a3c693f97e5a"
format_stamp: "Formatted at 2026-08-08 18:31:40 on dist-test-slave-jztv"
server_key: "ecf292915be9f95790e31988ace25367"
server_key_iv: "42026b88cec93370564fbf872ec6e8d9"
server_key_version: "encryptionkey@0"
I20260808 18:31:40.462852 3667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "7a94b818eb7843ca9e99a3c693f97e5a"
format_stamp: "Formatted at 2026-08-08 18:31:40 on dist-test-slave-jztv"
server_key: "ecf292915be9f95790e31988ace25367"
server_key_iv: "42026b88cec93370564fbf872ec6e8d9"
server_key_version: "encryptionkey@0"
I20260808 18:31:40.466714 3667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 18:31:40.469193 3683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:40.470479 3667 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:31:40.470608 3667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "7a94b818eb7843ca9e99a3c693f97e5a"
format_stamp: "Formatted at 2026-08-08 18:31:40 on dist-test-slave-jztv"
server_key: "ecf292915be9f95790e31988ace25367"
server_key_iv: "42026b88cec93370564fbf872ec6e8d9"
server_key_version: "encryptionkey@0"
I20260808 18:31:40.470723 3667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:40.487476 3667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:40.490599 3667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:40.490799 3667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:40.491384 3667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:40.492199 3667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:40.492246 3667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:40.492311 3667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:40.492343 3667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:40.501272 3667 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:41483
I20260808 18:31:40.501295 3838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:41483 every 8 connection(s)
I20260808 18:31:40.502324 3667 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:31:40.503566 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 3667
I20260808 18:31:40.503719 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:31:40.504031 7825 external_mini_cluster.cc:1428] Setting key c6d8b8bb71c3d37dbac933a286c8794d
Aug 08 18:31:40 dist-test-slave-jztv krb5kdc[3114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213900, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.67@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:31:40.516095 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.504274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:32891 (local address 127.7.164.126:45927)
0808 18:31:40.504431 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:40.504435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:40.505416 (+ 981us) server_negotiation.cc:407] Connection header received
0808 18:31:40.506860 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:40.506862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:40.506909 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:40.506977 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:40.508125 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.508670 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.509775 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.510012 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.510098 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.512589 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:40.512593 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:40.512596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:40.512627 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:40.513877 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:40.514523 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:40.514528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:40.514532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:40.514620 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:40.514981 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:40.514985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:40.514986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:40.515178 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:40.515292 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:40.515785 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 18:31:40.515909 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:31:40.517315 3842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.504743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:45927 (local address 127.7.164.67:32891)
0808 18:31:40.505254 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:40.505307 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:40.506665 (+ 1358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:40.507142 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:40.507149 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:40.507496 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:40.507961 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.507975 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.508784 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.508786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.509644 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.509653 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.510147 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.510150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.510364 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.510892 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:31:40.510908 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:40.512409 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:40.514036 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:40.514043 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:40.514058 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:40.514411 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:40.514722 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:31:40.514725 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:31:40.514728 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:31:40.514871 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:31:40.515291 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:40.515298 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:40.515539 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 18:31:40.515724 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 18:31:40.516037 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":157,"threads_started":1}
I20260808 18:31:40.518530 3839 heartbeater.cc:347] Connected to a master server at 127.7.164.126:45927
I20260808 18:31:40.518740 3839 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:40.519187 3839 heartbeater.cc:511] Master 127.7.164.126:45927 requested a full tablet report, sending...
I20260808 18:31:40.520188 3185 ts_manager.cc:194] Registered new tserver with Master: 7a94b818eb7843ca9e99a3c693f97e5a (127.7.164.67:41483)
I20260808 18:31:40.520767 3185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.164.67@KRBTEST.COM'} at 127.7.164.67:32891
I20260808 18:31:40.529707 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:31:40.539193 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.531740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57216 (local address 127.7.164.126:45927)
0808 18:31:40.531917 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:40.531920 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:40.532004 (+ 84us) server_negotiation.cc:407] Connection header received
0808 18:31:40.532103 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:40.532105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:40.532145 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:40.532210 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:40.532887 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.533388 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.534168 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.534442 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.534534 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.536004 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:40.536007 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:40.536008 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:40.536030 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:40.537291 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:40.537759 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:40.537766 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:40.537768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:40.537855 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:40.538157 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:40.538164 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:40.538168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:40.538484 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:40.538641 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:40.538894 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 18:31:40.539026 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 18:31:40.542116 3185 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:31:40.545277 3185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:31:40.560799 3851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.554144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:41483 (local address 127.0.0.1:33208)
0808 18:31:40.554985 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:40.555038 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:40.555190 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:40.555850 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:40.555856 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:40.555880 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:40.556028 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.556039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.557582 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.557585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.558907 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.558917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.560166 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.560170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.560299 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.560363 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 18:31:40.560473 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 18:31:40.560614 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":70,"spinlock_wait_cycles":9984,"thread_start_us":144,"threads_started":1}
I20260808 18:31:40.561081 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.554102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:44993 (local address 127.0.0.1:40670)
0808 18:31:40.554985 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:40.555041 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:40.555189 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:40.556560 (+ 1371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:40.556563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:40.556581 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:40.556691 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.556700 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.557915 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.557919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.559280 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.559291 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.560634 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.560640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.560774 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.560801 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:31:40.560875 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 18:31:40.560945 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":63,"spinlock_wait_cycles":12928,"thread_start_us":76,"threads_started":1}
I20260808 18:31:40.561308 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.554939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33208 (local address 127.7.164.67:41483)
0808 18:31:40.555385 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:40.555397 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:40.555420 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:31:40.555504 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:40.555509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:40.555704 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:40.555831 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:40.556178 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.557419 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.559044 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.559835 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.559944 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.560096 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:40.561015 (+ 919us) server_negotiation.cc:299] Negotiation successful
0808 18:31:40.561127 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":137,"threads_started":1}
I20260808 18:31:40.561290 3856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.554743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32938 (local address 127.7.164.65:43937)
0808 18:31:40.555617 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:40.555635 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:40.555659 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:31:40.555733 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:40.555738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:40.555940 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:40.556114 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:40.556384 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.557487 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.559379 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.560221 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.560323 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.560460 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:40.560826 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 18:31:40.561140 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":772,"thread_start_us":98,"threads_started":1}
I20260808 18:31:40.561730 3852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.554483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:43937 (local address 127.0.0.1:32938)
0808 18:31:40.555422 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:40.555435 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:40.555513 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:40.556121 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:40.556124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:40.556146 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:40.556256 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.556263 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.557637 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.557641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.559229 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.559240 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.560524 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.560527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.560655 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.560677 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 18:31:40.560752 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 18:31:40.560835 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"mutex_wait_us":270,"thread_start_us":84,"threads_started":1}
I20260808 18:31:40.562143 3854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.554247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.7.164.66:44993)
0808 18:31:40.556139 (+ 1892us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:40.556150 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:40.556172 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:31:40.556243 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:40.556248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:40.556429 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:40.556580 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:40.556836 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.557638 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.559427 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.560248 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.560376 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.560561 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:40.561921 (+ 1360us) server_negotiation.cc:299] Negotiation successful
0808 18:31:40.562026 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1755,"thread_start_us":126,"threads_started":1}
I20260808 18:31:40.564359 3362 tablet_service.cc:1467] Processing CreateTablet for tablet 1bb9db37131f496085d01891cb753647 (DEFAULT_TABLE table=test-table [id=27d16d5526ef4c3f8a5448724cc7a243]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:40.564455 3751 tablet_service.cc:1467] Processing CreateTablet for tablet 1bb9db37131f496085d01891cb753647 (DEFAULT_TABLE table=test-table [id=27d16d5526ef4c3f8a5448724cc7a243]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:40.564621 3549 tablet_service.cc:1467] Processing CreateTablet for tablet 1bb9db37131f496085d01891cb753647 (DEFAULT_TABLE table=test-table [id=27d16d5526ef4c3f8a5448724cc7a243]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:40.565399 3362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bb9db37131f496085d01891cb753647. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:40.565524 3751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bb9db37131f496085d01891cb753647. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:40.565517 3549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bb9db37131f496085d01891cb753647. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:40.571558 3858 tablet_bootstrap.cc:492] T 1bb9db37131f496085d01891cb753647 P def4da0e5a414521a5bedad8218611ae: Bootstrap starting.
I20260808 18:31:40.573314 3859 tablet_bootstrap.cc:492] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47: Bootstrap starting.
I20260808 18:31:40.574440 3858 tablet_bootstrap.cc:654] T 1bb9db37131f496085d01891cb753647 P def4da0e5a414521a5bedad8218611ae: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:40.574604 3860 tablet_bootstrap.cc:492] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a: Bootstrap starting.
I20260808 18:31:40.575284 3858 log.cc:824] T 1bb9db37131f496085d01891cb753647 P def4da0e5a414521a5bedad8218611ae: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:40.575997 3859 tablet_bootstrap.cc:654] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:40.576778 3860 tablet_bootstrap.cc:654] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:40.576844 3859 log.cc:824] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:40.577149 3858 tablet_bootstrap.cc:492] T 1bb9db37131f496085d01891cb753647 P def4da0e5a414521a5bedad8218611ae: No bootstrap required, opened a new log
I20260808 18:31:40.577366 3858 ts_tablet_manager.cc:1397] T 1bb9db37131f496085d01891cb753647 P def4da0e5a414521a5bedad8218611ae: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260808 18:31:40.578094 3860 log.cc:824] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:40.579048 3859 tablet_bootstrap.cc:492] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47: No bootstrap required, opened a new log
I20260808 18:31:40.579253 3859 ts_tablet_manager.cc:1397] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:31:40.581341 3858 raft_consensus.cc:348] T 1bb9db37131f496085d01891cb753647 P def4da0e5a414521a5bedad8218611ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eceb9e9fad444679aa883899ad40ec47" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44993 } } peers { permanent_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41483 } } peers { permanent_uuid: "def4da0e5a414521a5bedad8218611ae" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 43937 } }
I20260808 18:31:40.581555 3858 raft_consensus.cc:374] T 1bb9db37131f496085d01891cb753647 P def4da0e5a414521a5bedad8218611ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:40.581589 3858 raft_consensus.cc:729] T 1bb9db37131f496085d01891cb753647 P def4da0e5a414521a5bedad8218611ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: def4da0e5a414521a5bedad8218611ae, State: Initialized, Role: FOLLOWER
I20260808 18:31:40.582034 3858 consensus_queue.cc:260] T 1bb9db37131f496085d01891cb753647 P def4da0e5a414521a5bedad8218611ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eceb9e9fad444679aa883899ad40ec47" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44993 } } peers { permanent_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41483 } } peers { permanent_uuid: "def4da0e5a414521a5bedad8218611ae" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 43937 } }
I20260808 18:31:40.582261 3859 raft_consensus.cc:348] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eceb9e9fad444679aa883899ad40ec47" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44993 } } peers { permanent_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41483 } } peers { permanent_uuid: "def4da0e5a414521a5bedad8218611ae" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 43937 } }
I20260808 18:31:40.582610 3859 raft_consensus.cc:374] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:40.582690 3452 heartbeater.cc:503] Master 127.7.164.126:45927 was elected leader, sending a full tablet report...
I20260808 18:31:40.582721 3859 raft_consensus.cc:729] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eceb9e9fad444679aa883899ad40ec47, State: Initialized, Role: FOLLOWER
I20260808 18:31:40.583159 3859 consensus_queue.cc:260] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eceb9e9fad444679aa883899ad40ec47" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44993 } } peers { permanent_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41483 } } peers { permanent_uuid: "def4da0e5a414521a5bedad8218611ae" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 43937 } }
I20260808 18:31:40.583820 3657 heartbeater.cc:503] Master 127.7.164.126:45927 was elected leader, sending a full tablet report...
I20260808 18:31:40.584410 3860 tablet_bootstrap.cc:492] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a: No bootstrap required, opened a new log
I20260808 18:31:40.584630 3860 ts_tablet_manager.cc:1397] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260808 18:31:40.584916 3858 ts_tablet_manager.cc:1428] T 1bb9db37131f496085d01891cb753647 P def4da0e5a414521a5bedad8218611ae: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260808 18:31:40.585700 3859 ts_tablet_manager.cc:1428] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:31:40.588516 3860 raft_consensus.cc:348] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eceb9e9fad444679aa883899ad40ec47" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44993 } } peers { permanent_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41483 } } peers { permanent_uuid: "def4da0e5a414521a5bedad8218611ae" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 43937 } }
I20260808 18:31:40.588799 3860 raft_consensus.cc:374] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:40.588874 3860 raft_consensus.cc:729] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a94b818eb7843ca9e99a3c693f97e5a, State: Initialized, Role: FOLLOWER
I20260808 18:31:40.589427 3860 consensus_queue.cc:260] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eceb9e9fad444679aa883899ad40ec47" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44993 } } peers { permanent_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41483 } } peers { permanent_uuid: "def4da0e5a414521a5bedad8218611ae" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 43937 } }
I20260808 18:31:40.590149 3839 heartbeater.cc:503] Master 127.7.164.126:45927 was elected leader, sending a full tablet report...
I20260808 18:31:40.590497 3860 ts_tablet_manager.cc:1428] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:31:40.603659 3867 raft_consensus.cc:482] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:31:40.603855 3867 raft_consensus.cc:504] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eceb9e9fad444679aa883899ad40ec47" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44993 } } peers { permanent_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41483 } } peers { permanent_uuid: "def4da0e5a414521a5bedad8218611ae" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 43937 } }
I20260808 18:31:40.604878 3867 leader_election.cc:288] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eceb9e9fad444679aa883899ad40ec47 (127.7.164.66:44993), def4da0e5a414521a5bedad8218611ae (127.7.164.65:43937)
I20260808 18:31:40.608981 3842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.605140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:43937 (local address 127.7.164.67:54581)
0808 18:31:40.605256 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:40.605274 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:40.605363 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:40.605643 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:40.605645 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:40.605664 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:40.605834 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.605843 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.606883 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.606887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.607802 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.607813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.608554 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.608558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.608668 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.608706 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 18:31:40.608784 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 18:31:40.608863 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":38}
I20260808 18:31:40.609021 3856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.605264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:54581 (local address 127.7.164.65:43937)
0808 18:31:40.605384 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:40.605389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:40.605409 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:31:40.605473 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:40.605478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:40.605527 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:40.605625 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:40.605992 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.606757 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.607940 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.608425 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.608522 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.608565 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:40.608814 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 18:31:40.608890 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:31:40.609269 3868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.605140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:44993 (local address 127.7.164.67:33715)
0808 18:31:40.605645 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:40.605661 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:40.605772 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:40.606109 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:40.606113 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:40.606134 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:40.606287 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.606295 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.607120 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.607123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.608034 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:40.608044 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.608808 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.608812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:40.608928 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.608953 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:31:40.609035 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 18:31:40.609110 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":137,"thread_start_us":150,"threads_started":1}
I20260808 18:31:40.609314 3854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.605243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:33715 (local address 127.7.164.66:44993)
0808 18:31:40.605384 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:40.605389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:40.605757 (+ 368us) server_negotiation.cc:407] Connection header received
0808 18:31:40.605889 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:40.605893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:40.605966 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:40.606038 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:40.606417 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.606996 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.608169 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.608654 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.608782 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.608827 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:40.609078 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 18:31:40.609164 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:31:40.609882 3394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bb9db37131f496085d01891cb753647" candidate_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "def4da0e5a414521a5bedad8218611ae" is_pre_election: true
I20260808 18:31:40.610126 3581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bb9db37131f496085d01891cb753647" candidate_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eceb9e9fad444679aa883899ad40ec47" is_pre_election: true
I20260808 18:31:40.610262 3394 raft_consensus.cc:2454] T 1bb9db37131f496085d01891cb753647 P def4da0e5a414521a5bedad8218611ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a94b818eb7843ca9e99a3c693f97e5a in term 0.
I20260808 18:31:40.610416 3581 raft_consensus.cc:2454] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a94b818eb7843ca9e99a3c693f97e5a in term 0.
I20260808 18:31:40.610821 3692 leader_election.cc:302] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7a94b818eb7843ca9e99a3c693f97e5a, def4da0e5a414521a5bedad8218611ae, eceb9e9fad444679aa883899ad40ec47; no voters:
I20260808 18:31:40.611093 3867 raft_consensus.cc:2781] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:31:40.611181 3867 raft_consensus.cc:482] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:31:40.611225 3867 raft_consensus.cc:3037] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:40.612301 3867 raft_consensus.cc:504] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eceb9e9fad444679aa883899ad40ec47" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44993 } } peers { permanent_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41483 } } peers { permanent_uuid: "def4da0e5a414521a5bedad8218611ae" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 43937 } }
I20260808 18:31:40.612645 3867 leader_election.cc:288] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [CANDIDATE]: Term 1 election: Requested vote from peers eceb9e9fad444679aa883899ad40ec47 (127.7.164.66:44993), def4da0e5a414521a5bedad8218611ae (127.7.164.65:43937)
I20260808 18:31:40.613072 3394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bb9db37131f496085d01891cb753647" candidate_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "def4da0e5a414521a5bedad8218611ae"
I20260808 18:31:40.613081 3581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bb9db37131f496085d01891cb753647" candidate_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eceb9e9fad444679aa883899ad40ec47"
I20260808 18:31:40.613220 3394 raft_consensus.cc:3037] T 1bb9db37131f496085d01891cb753647 P def4da0e5a414521a5bedad8218611ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:40.613241 3581 raft_consensus.cc:3037] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:40.614400 3394 raft_consensus.cc:2454] T 1bb9db37131f496085d01891cb753647 P def4da0e5a414521a5bedad8218611ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a94b818eb7843ca9e99a3c693f97e5a in term 1.
I20260808 18:31:40.614405 3581 raft_consensus.cc:2454] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a94b818eb7843ca9e99a3c693f97e5a in term 1.
I20260808 18:31:40.614820 3691 leader_election.cc:302] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7a94b818eb7843ca9e99a3c693f97e5a, def4da0e5a414521a5bedad8218611ae, eceb9e9fad444679aa883899ad40ec47; no voters:
I20260808 18:31:40.615017 3867 raft_consensus.cc:2781] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:40.615263 3867 raft_consensus.cc:686] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [term 1 LEADER]: Becoming Leader. State: Replica: 7a94b818eb7843ca9e99a3c693f97e5a, State: Running, Role: LEADER
I20260808 18:31:40.615619 3867 consensus_queue.cc:237] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eceb9e9fad444679aa883899ad40ec47" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44993 } } peers { permanent_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41483 } } peers { permanent_uuid: "def4da0e5a414521a5bedad8218611ae" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 43937 } }
I20260808 18:31:40.618798 3182 catalog_manager.cc:5495] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a94b818eb7843ca9e99a3c693f97e5a (127.7.164.67). New cstate: current_term: 1 leader_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eceb9e9fad444679aa883899ad40ec47" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a94b818eb7843ca9e99a3c693f97e5a" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "def4da0e5a414521a5bedad8218611ae" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 43937 } health_report { overall_health: UNKNOWN } } }
I20260808 18:31:40.630594 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.627376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33224 (local address 127.7.164.67:41483)
0808 18:31:40.627499 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:40.627503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:40.627518 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:31:40.627634 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:40.627637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:40.627688 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:40.627775 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:31:40.627997 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.628484 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.629283 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.629485 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.629561 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.629815 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:31:40.630264 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:31:40.630363 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:40.630421 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 18:31:40.630468 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:31:40.636492 3581 raft_consensus.cc:1261] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47 [term 1 FOLLOWER]: Refusing update from remote peer 7a94b818eb7843ca9e99a3c693f97e5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:31:40.636490 3394 raft_consensus.cc:1261] T 1bb9db37131f496085d01891cb753647 P def4da0e5a414521a5bedad8218611ae [term 1 FOLLOWER]: Refusing update from remote peer 7a94b818eb7843ca9e99a3c693f97e5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:31:40.637173 3867 consensus_queue.cc:1035] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "eceb9e9fad444679aa883899ad40ec47" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 44993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:31:40.637377 3870 consensus_queue.cc:1035] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "def4da0e5a414521a5bedad8218611ae" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 43937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:31:40.644164 3874 mvcc.cc:205] Tried to move back new op lower bound from 7316332136999514112 to 7316332136924311552. Current Snapshot: MvccSnapshot[applied={T|T < 7316332136999514112}]
I20260808 18:31:40.646803 3875 mvcc.cc:205] Tried to move back new op lower bound from 7316332136999514112 to 7316332136924311552. Current Snapshot: MvccSnapshot[applied={T|T < 7316332136999514112}]
I20260808 18:31:40.654145 3182 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:31:40.654358 3182 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:31:40.656878 3182 catalog_manager.cc:5782] T 00000000000000000000000000000000 P adb95dadff1d47199fe166ceb7e1cd16: Sending DeleteTablet for 3 replicas of tablet 1bb9db37131f496085d01891cb753647
I20260808 18:31:40.657627 3549 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bb9db37131f496085d01891cb753647 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:40 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:40670
I20260808 18:31:40.657994 3362 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bb9db37131f496085d01891cb753647 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:40 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:32938
I20260808 18:31:40.658177 3885 tablet_replica.cc:330] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47: stopping tablet replica
I20260808 18:31:40.658171 3751 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bb9db37131f496085d01891cb753647 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:40 UTC) from {username='kudu', principal='kudu/127.7.164.126@KRBTEST.COM'} at 127.0.0.1:33208
I20260808 18:31:40.658526 3885 raft_consensus.cc:2229] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:31:40.658638 3889 tablet_replica.cc:330] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a: stopping tablet replica
I20260808 18:31:40.658893 3885 raft_consensus.cc:2258] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:31:40.658941 3889 raft_consensus.cc:2229] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:31:40.659338 3889 raft_consensus.cc:2258] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:31:40.659314 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 3269
I20260808 18:31:40.661685 3889 ts_tablet_manager.cc:1905] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:31:40.665792 3889 ts_tablet_manager.cc:1918] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:31:40.665977 3889 log.cc:1196] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/1bb9db37131f496085d01891cb753647
I20260808 18:31:40.666520 3889 ts_tablet_manager.cc:1939] T 1bb9db37131f496085d01891cb753647 P 7a94b818eb7843ca9e99a3c693f97e5a: Deleting consensus metadata
I20260808 18:31:40.668339 3161 catalog_manager.cc:4841] TS 7a94b818eb7843ca9e99a3c693f97e5a (127.7.164.67:41483): tablet 1bb9db37131f496085d01891cb753647 (table test-table [id=27d16d5526ef4c3f8a5448724cc7a243]) successfully deleted
I20260808 18:31:40.674337 3885 ts_tablet_manager.cc:1905] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:31:40.677338 3885 ts_tablet_manager.cc:1918] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:31:40.677578 3885 log.cc:1196] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/1bb9db37131f496085d01891cb753647
I20260808 18:31:40.678012 3885 ts_tablet_manager.cc:1939] T 1bb9db37131f496085d01891cb753647 P eceb9e9fad444679aa883899ad40ec47: Deleting consensus metadata
I20260808 18:31:40.679530 3163 catalog_manager.cc:4841] TS eceb9e9fad444679aa883899ad40ec47 (127.7.164.66:44993): tablet 1bb9db37131f496085d01891cb753647 (table test-table [id=27d16d5526ef4c3f8a5448724cc7a243]) successfully deleted
W20260808 18:31:40.679854 3164 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:43937 (error 108)
I20260808 18:31:40.680049 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 3458
I20260808 18:31:40.694512 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 3667
I20260808 18:31:40.695993 3852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.695570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:43937 (local address 127.0.0.1:32944)
0808 18:31:40.695791 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:40.695884 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:43937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260808 18:31:40.696282 3164 catalog_manager.cc:4584] TS def4da0e5a414521a5bedad8218611ae (127.7.164.65:43937): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1bb9db37131f496085d01891cb753647: Network error: Client connection negotiation failed: client connection to 127.7.164.65:43937: connect: Connection refused (error 111)
I20260808 18:31:40.705973 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 3134
2026-08-08T18:31:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2568 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T18:31:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:40Z Disabled control of system clock
I20260808 18:31:40.744043 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:35307
--webserver_interface=127.7.164.126
--webserver_port=0
--builtin_ntp_servers=127.7.164.84:35443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:35307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:31:40.857925 3894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:40.858206 3894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:40.858352 3894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:40.862485 3894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:40.862560 3894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:40.862577 3894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:40.862594 3894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:40.862610 3894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:40.867609 3894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:35443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:35307
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:35307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:40.868768 3894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:40.870460 3894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:40.879195 3901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:40.879242 3904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:40.879320 3894 server_base.cc:1034] running on GCE node
W20260808 18:31:40.879241 3902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:40.880108 3894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:40.881137 3894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:40.882296 3894 hybrid_clock.cc:648] HybridClock initialized: now 1786213900882278 us; error 27 us; skew 500 ppm
I20260808 18:31:40.884099 3894 webserver.cc:460] Webserver started at http://127.7.164.126:33993/ using document root <none> and password file <none>
I20260808 18:31:40.884680 3894 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:40.884742 3894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:40.885010 3894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:40.887158 3894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "a86f9e7694564446b754bfbcd9c44ec5"
format_stamp: "Formatted at 2026-08-08 18:31:40 on dist-test-slave-jztv"
server_key: "e3acf5ef8a7a2f04fafc8ab9aa914f2b"
server_key_iv: "29162af1806b43d6be6a56bf43e7875e"
server_key_version: "encryptionkey@0"
I20260808 18:31:40.887743 3894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "a86f9e7694564446b754bfbcd9c44ec5"
format_stamp: "Formatted at 2026-08-08 18:31:40 on dist-test-slave-jztv"
server_key: "e3acf5ef8a7a2f04fafc8ab9aa914f2b"
server_key_iv: "29162af1806b43d6be6a56bf43e7875e"
server_key_version: "encryptionkey@0"
I20260808 18:31:40.891494 3894 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:31:40.893888 3913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:40.895144 3894 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:31:40.895303 3894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "a86f9e7694564446b754bfbcd9c44ec5"
format_stamp: "Formatted at 2026-08-08 18:31:40 on dist-test-slave-jztv"
server_key: "e3acf5ef8a7a2f04fafc8ab9aa914f2b"
server_key_iv: "29162af1806b43d6be6a56bf43e7875e"
server_key_version: "encryptionkey@0"
I20260808 18:31:40.895424 3894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:40.919499 3894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:40.920254 3894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:40.920472 3894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:40.927721 3894 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:35307
I20260808 18:31:40.927733 3986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:35307 every 8 connection(s)
I20260808 18:31:40.928663 3894 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:40.930363 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 3894
I20260808 18:31:40.930506 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:40.930831 7825 external_mini_cluster.cc:1428] Setting key c986dfc5a050052ed0d6a09380bb6501
I20260808 18:31:40.932193 3988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:40.939424 3988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5: Bootstrap starting.
I20260808 18:31:40.941697 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:40.932287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53682 (local address 127.7.164.126:35307)
0808 18:31:40.932985 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:40.933007 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:40.933053 (+ 46us) server_negotiation.cc:407] Connection header received
0808 18:31:40.933990 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:40.934015 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:40.934589 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:40.934974 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:40.935431 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.936922 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.937801 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:40.938561 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:40.938722 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:40.939608 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:40.939614 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:40.939630 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:40.940121 (+ 491us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:40.940198 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:40.940225 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:40.940326 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:40.940471 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 18:31:40.940771 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":139,"threads_started":1}
I20260808 18:31:40.942237 3988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:40.943084 3988 log.cc:824] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:40.945106 3988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5: No bootstrap required, opened a new log
I20260808 18:31:40.948624 3988 raft_consensus.cc:348] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86f9e7694564446b754bfbcd9c44ec5" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35307 } }
I20260808 18:31:40.948835 3988 raft_consensus.cc:374] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:40.948927 3988 raft_consensus.cc:729] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a86f9e7694564446b754bfbcd9c44ec5, State: Initialized, Role: FOLLOWER
I20260808 18:31:40.949358 3988 consensus_queue.cc:260] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86f9e7694564446b754bfbcd9c44ec5" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35307 } }
I20260808 18:31:40.949510 3988 raft_consensus.cc:388] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:40.949580 3988 raft_consensus.cc:482] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:40.949677 3988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:40.950734 3988 raft_consensus.cc:504] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86f9e7694564446b754bfbcd9c44ec5" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35307 } }
I20260808 18:31:40.951041 3988 leader_election.cc:302] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a86f9e7694564446b754bfbcd9c44ec5; no voters:
I20260808 18:31:40.951314 3988 leader_election.cc:288] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:40.951460 3998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:40.951735 3998 raft_consensus.cc:686] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [term 1 LEADER]: Becoming Leader. State: Replica: a86f9e7694564446b754bfbcd9c44ec5, State: Running, Role: LEADER
I20260808 18:31:40.952081 3998 consensus_queue.cc:237] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86f9e7694564446b754bfbcd9c44ec5" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35307 } }
I20260808 18:31:40.952661 3988 sys_catalog.cc:564] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:40.953688 3999 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a86f9e7694564446b754bfbcd9c44ec5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86f9e7694564446b754bfbcd9c44ec5" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35307 } } }
I20260808 18:31:40.953837 3999 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:40.954188 4000 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a86f9e7694564446b754bfbcd9c44ec5. Latest consensus state: current_term: 1 leader_uuid: "a86f9e7694564446b754bfbcd9c44ec5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86f9e7694564446b754bfbcd9c44ec5" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35307 } } }
I20260808 18:31:40.954403 4000 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5 [sys.catalog]: This master's current role is: LEADER
W20260808 18:31:40.957999 4015 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:31:40.958069 4015 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:31:40.958155 4007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:40.958853 4007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:40.964407 4007 catalog_manager.cc:1347] Generated new cluster ID: 6f80b5b5b668499ca41b01b912831d2c
I20260808 18:31:40.964478 4007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:40.978153 4007 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:40.979118 4007 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:40.983271 4007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5: Generated new TSK 0
I20260808 18:31:40.984014 4007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:31:40.996239 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.65:0
--local_ip_for_outbound_sockets=127.7.164.65
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35307
--builtin_ntp_servers=127.7.164.84:35443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:31:41.110144 4022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:41.110467 4022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:41.110535 4022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:41.114655 4022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:41.114719 4022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:41.114813 4022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.65
I20260808 18:31:41.119731 4022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:35443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.164.65
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.65
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:41.120846 4022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:41.122270 4022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:41.132011 4029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:41.132059 4028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:41.132074 4022 server_base.cc:1034] running on GCE node
W20260808 18:31:41.132037 4031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:41.132725 4022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:41.133320 4022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:41.134483 4022 hybrid_clock.cc:648] HybridClock initialized: now 1786213901134449 us; error 45 us; skew 500 ppm
I20260808 18:31:41.136451 4022 webserver.cc:460] Webserver started at http://127.7.164.65:42515/ using document root <none> and password file <none>
I20260808 18:31:41.137133 4022 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:41.137199 4022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:41.137437 4022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:41.139616 4022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/data/instance:
uuid: "0d59686e57a74ed7ad2f081bbfd38b17"
format_stamp: "Formatted at 2026-08-08 18:31:41 on dist-test-slave-jztv"
server_key: "373c0d023c0e3e233fe3554d54668ee8"
server_key_iv: "bc0b3d12aac16f27ce5e2e7bd7d8164c"
server_key_version: "encryptionkey@0"
I20260808 18:31:41.140189 4022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance:
uuid: "0d59686e57a74ed7ad2f081bbfd38b17"
format_stamp: "Formatted at 2026-08-08 18:31:41 on dist-test-slave-jztv"
server_key: "373c0d023c0e3e233fe3554d54668ee8"
server_key_iv: "bc0b3d12aac16f27ce5e2e7bd7d8164c"
server_key_version: "encryptionkey@0"
I20260808 18:31:41.144009 4022 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260808 18:31:41.146589 4041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:41.147838 4022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260808 18:31:41.147984 4022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/wal
uuid: "0d59686e57a74ed7ad2f081bbfd38b17"
format_stamp: "Formatted at 2026-08-08 18:31:41 on dist-test-slave-jztv"
server_key: "373c0d023c0e3e233fe3554d54668ee8"
server_key_iv: "bc0b3d12aac16f27ce5e2e7bd7d8164c"
server_key_version: "encryptionkey@0"
I20260808 18:31:41.148110 4022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:41.167563 4022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:41.168336 4022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:41.168572 4022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:41.169209 4022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:41.170207 4022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:41.170305 4022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:41.170375 4022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:41.170406 4022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:41.180174 4022 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.65:33099
I20260808 18:31:41.180208 4209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.65:33099 every 8 connection(s)
I20260808 18:31:41.181120 4022 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/data/info.pb
I20260808 18:31:41.183928 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 4022
I20260808 18:31:41.184068 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-0/wal/instance
I20260808 18:31:41.184388 7825 external_mini_cluster.cc:1428] Setting key 1d1627281624140915c97f677e4ca4c2
I20260808 18:31:41.186470 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.66:0
--local_ip_for_outbound_sockets=127.7.164.66
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35307
--builtin_ntp_servers=127.7.164.84:35443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:31:41.191175 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.183089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.65:49505 (local address 127.7.164.126:35307)
0808 18:31:41.183245 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:41.183250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:41.184485 (+ 1235us) server_negotiation.cc:407] Connection header received
0808 18:31:41.186152 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:41.186155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:41.186239 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:41.186320 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:41.187332 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.187871 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.189051 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.189291 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.189410 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.190634 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:41.190638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:41.190641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:41.190655 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:41.190682 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:41.190690 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:41.190770 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:41.190947 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 18:31:41.191025 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:31:41.192047 4212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.183552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35307 (local address 127.7.164.65:49505)
0808 18:31:41.184314 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:41.184368 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:41.185942 (+ 1574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:41.186521 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:41.186528 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:41.186647 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:41.187181 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.187193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.188016 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.188020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.188912 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.188921 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.189454 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.189466 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.189634 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.190352 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:31:41.190354 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:41.190397 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:31:41.190470 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:41.190784 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:41.190796 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:41.190805 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 18:31:41.190945 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 18:31:41.191237 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":144,"threads_started":1}
I20260808 18:31:41.193513 4210 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35307
I20260808 18:31:41.193771 4210 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:41.194389 4210 heartbeater.cc:511] Master 127.7.164.126:35307 requested a full tablet report, sending...
I20260808 18:31:41.195974 3943 ts_manager.cc:194] Registered new tserver with Master: 0d59686e57a74ed7ad2f081bbfd38b17 (127.7.164.65:33099)
I20260808 18:31:41.197131 3943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.164.65:49505
W20260808 18:31:41.303880 4213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:41.304167 4213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:41.304245 4213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:41.308276 4213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:41.308364 4213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:41.308454 4213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.66
I20260808 18:31:41.313306 4213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:35443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.164.66
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.66
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:41.314471 4213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:41.315855 4213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:41.325616 4223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:41.325656 4213 server_base.cc:1034] running on GCE node
W20260808 18:31:41.325603 4221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:41.325585 4220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:41.326365 4213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:41.326998 4213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:41.328195 4213 hybrid_clock.cc:648] HybridClock initialized: now 1786213901328182 us; error 25 us; skew 500 ppm
I20260808 18:31:41.329999 4213 webserver.cc:460] Webserver started at http://127.7.164.66:33095/ using document root <none> and password file <none>
I20260808 18:31:41.330662 4213 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:41.330727 4213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:41.330992 4213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:41.333101 4213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/data/instance:
uuid: "65d6d8944cbf429db731603bf2753ba6"
format_stamp: "Formatted at 2026-08-08 18:31:41 on dist-test-slave-jztv"
server_key: "80d8aee7afeaf1dad2dd0402053268e7"
server_key_iv: "150bddedfb39659fcde54b037dcb2d09"
server_key_version: "encryptionkey@0"
I20260808 18:31:41.333683 4213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance:
uuid: "65d6d8944cbf429db731603bf2753ba6"
format_stamp: "Formatted at 2026-08-08 18:31:41 on dist-test-slave-jztv"
server_key: "80d8aee7afeaf1dad2dd0402053268e7"
server_key_iv: "150bddedfb39659fcde54b037dcb2d09"
server_key_version: "encryptionkey@0"
I20260808 18:31:41.337556 4213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 18:31:41.340099 4230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:41.341269 4213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:31:41.341423 4213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/wal
uuid: "65d6d8944cbf429db731603bf2753ba6"
format_stamp: "Formatted at 2026-08-08 18:31:41 on dist-test-slave-jztv"
server_key: "80d8aee7afeaf1dad2dd0402053268e7"
server_key_iv: "150bddedfb39659fcde54b037dcb2d09"
server_key_version: "encryptionkey@0"
I20260808 18:31:41.341549 4213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:41.354305 4213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:41.355115 4213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:41.355325 4213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:41.355935 4213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:41.356931 4213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:41.357016 4213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:41.357108 4213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:41.357157 4213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:41.366539 4213 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.66:39367
I20260808 18:31:41.366585 4395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.66:39367 every 8 connection(s)
I20260808 18:31:41.367493 4213 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/data/info.pb
I20260808 18:31:41.374269 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 4213
I20260808 18:31:41.374387 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/wal/instance
I20260808 18:31:41.374704 7825 external_mini_cluster.cc:1428] Setting key aaf284cd85c0dbf0f8f72e282f1842cd
I20260808 18:31:41.376890 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.164.67:0
--local_ip_for_outbound_sockets=127.7.164.67
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35307
--builtin_ntp_servers=127.7.164.84:35443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:31:41.377856 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.369456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.66:35199 (local address 127.7.164.126:35307)
0808 18:31:41.369648 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:41.369653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:41.370804 (+ 1151us) server_negotiation.cc:407] Connection header received
0808 18:31:41.372227 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:41.372229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:41.372281 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:41.372364 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:41.373366 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.373883 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.375474 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.375726 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.375814 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.377265 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:41.377271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:41.377277 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:41.377294 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:41.377318 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:41.377331 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:41.377410 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:41.377564 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 18:31:41.377638 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 18:31:41.378736 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.369850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35307 (local address 127.7.164.66:35199)
0808 18:31:41.370648 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:41.370695 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:41.372023 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:41.372612 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:41.372619 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:41.372705 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:41.373225 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.373242 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.374074 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.374077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.375359 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.375370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.375901 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.375905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.376155 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.376909 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:31:41.376911 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:41.376994 (+ 83us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:31:41.377097 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:41.377419 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:41.377427 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:41.377432 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 18:31:41.377534 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 18:31:41.377764 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":119,"threads_started":1}
I20260808 18:31:41.380043 4397 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35307
I20260808 18:31:41.380312 4397 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:41.380872 4397 heartbeater.cc:511] Master 127.7.164.126:35307 requested a full tablet report, sending...
I20260808 18:31:41.382123 3943 ts_manager.cc:194] Registered new tserver with Master: 65d6d8944cbf429db731603bf2753ba6 (127.7.164.66:39367)
I20260808 18:31:41.383139 3943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.164.66:35199
W20260808 18:31:41.491178 4401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:41.491425 4401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:41.491468 4401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:41.495306 4401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:41.495349 4401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:41.495400 4401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.164.67
I20260808 18:31:41.500094 4401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:35443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.164.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.164.67
--webserver_port=0
--tserver_master_addrs=127.7.164.126:35307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.164.67
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:41.501056 4401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:41.502339 4401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:41.511807 4407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:41.511741 4411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:41.511802 4408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:41.512126 4401 server_base.cc:1034] running on GCE node
I20260808 18:31:41.512571 4401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:41.513235 4401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:41.514428 4401 hybrid_clock.cc:648] HybridClock initialized: now 1786213901514391 us; error 56 us; skew 500 ppm
I20260808 18:31:41.516218 4401 webserver.cc:460] Webserver started at http://127.7.164.67:37833/ using document root <none> and password file <none>
I20260808 18:31:41.516793 4401 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:41.516889 4401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:41.517132 4401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:41.519260 4401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/data/instance:
uuid: "fd80e1ed40a141dd85ee7444cbd8e144"
format_stamp: "Formatted at 2026-08-08 18:31:41 on dist-test-slave-jztv"
server_key: "1e5e3c27255d4ae6503e14c0b2bc8dec"
server_key_iv: "e94a8cfef1e2fa597a117bb8e206fe92"
server_key_version: "encryptionkey@0"
I20260808 18:31:41.519851 4401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance:
uuid: "fd80e1ed40a141dd85ee7444cbd8e144"
format_stamp: "Formatted at 2026-08-08 18:31:41 on dist-test-slave-jztv"
server_key: "1e5e3c27255d4ae6503e14c0b2bc8dec"
server_key_iv: "e94a8cfef1e2fa597a117bb8e206fe92"
server_key_version: "encryptionkey@0"
I20260808 18:31:41.523506 4401 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:31:41.525894 4417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:41.527143 4401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:31:41.527287 4401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/wal
uuid: "fd80e1ed40a141dd85ee7444cbd8e144"
format_stamp: "Formatted at 2026-08-08 18:31:41 on dist-test-slave-jztv"
server_key: "1e5e3c27255d4ae6503e14c0b2bc8dec"
server_key_iv: "e94a8cfef1e2fa597a117bb8e206fe92"
server_key_version: "encryptionkey@0"
I20260808 18:31:41.527405 4401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:41.539060 4401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:41.539703 4401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:41.539897 4401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:41.540458 4401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:31:41.541378 4401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:31:41.541452 4401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:41.541517 4401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:31:41.541553 4401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:41.550374 4401 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.67:36305
I20260808 18:31:41.550447 4578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.67:36305 every 8 connection(s)
I20260808 18:31:41.551287 4401 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/data/info.pb
I20260808 18:31:41.554798 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 4401
I20260808 18:31:41.554934 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/wal/instance
I20260808 18:31:41.555233 7825 external_mini_cluster.cc:1428] Setting key 3474160d0f7760cc7a143eea9896a7c6
I20260808 18:31:41.560967 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.553269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:59843 (local address 127.7.164.126:35307)
0808 18:31:41.553438 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:41.553442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:41.554595 (+ 1153us) server_negotiation.cc:407] Connection header received
0808 18:31:41.556255 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:41.556258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:41.556325 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:41.556396 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:41.557491 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.558045 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.559237 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.559477 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.559562 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.560574 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:41.560577 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:41.560580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:41.560591 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:41.560609 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:41.560617 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:41.560671 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:41.560807 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 18:31:41.560855 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 18:31:41.561796 4581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.553645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:35307 (local address 127.7.164.67:59843)
0808 18:31:41.554401 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:41.554452 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:41.556015 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:41.556623 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:41.556630 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:41.556755 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:31:41.557322 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.557343 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.558168 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.558170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.559118 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.559128 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.559616 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.559619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.559766 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.560326 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:31:41.560327 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:31:41.560366 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:31:41.560427 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:31:41.560686 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:31:41.560693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:31:41.560697 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:31:41.560791 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 18:31:41.560999 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":143,"threads_started":1}
I20260808 18:31:41.562930 4579 heartbeater.cc:347] Connected to a master server at 127.7.164.126:35307
I20260808 18:31:41.563134 4579 heartbeater.cc:464] Registering TS with master...
I20260808 18:31:41.563622 4579 heartbeater.cc:511] Master 127.7.164.126:35307 requested a full tablet report, sending...
I20260808 18:31:41.564613 3943 ts_manager.cc:194] Registered new tserver with Master: fd80e1ed40a141dd85ee7444cbd8e144 (127.7.164.67:36305)
I20260808 18:31:41.565199 3943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.164.67:59843
I20260808 18:31:41.569214 7825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:31:41.574597 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.571149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53692 (local address 127.7.164.126:35307)
0808 18:31:41.571279 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:41.571282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:41.571497 (+ 215us) server_negotiation.cc:407] Connection header received
0808 18:31:41.571591 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:41.571593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:41.571632 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:41.571699 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:41.571923 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.572406 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.573168 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.573370 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.573445 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.574181 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:41.574184 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:41.574186 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:41.574197 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:41.574260 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:41.574272 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:41.574342 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:41.574441 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 18:31:41.574487 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 18:31:41.577306 3943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:31:41.580281 3943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:31:41.595273 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.589015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.67:36305 (local address 127.0.0.1:45420)
0808 18:31:41.590350 (+ 1335us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:41.590364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:41.590472 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:41.590890 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:41.590893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:41.590911 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:41.591020 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.591031 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.592152 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.592157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.593532 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.593542 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.594676 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.594680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.594790 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.594839 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 18:31:41.594977 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 18:31:41.595102 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1270,"mutex_wait_us":136,"thread_start_us":119,"threads_started":1}
I20260808 18:31:41.595312 4594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.589611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45420 (local address 127.7.164.67:36305)
0808 18:31:41.589995 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:41.590007 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:41.590475 (+ 468us) server_negotiation.cc:407] Connection header received
0808 18:31:41.590570 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:41.590578 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:41.590754 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:41.590912 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:41.591149 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.592038 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.593663 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.594504 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.594633 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.594832 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:41.595030 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 18:31:41.595146 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":165,"threads_started":1}
I20260808 18:31:41.595312 4591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.588903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:39367 (local address 127.0.0.1:54782)
0808 18:31:41.589398 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:41.589445 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:41.589591 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:41.590690 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:41.590696 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:41.590721 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:41.590864 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.590871 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.592257 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.592260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.593668 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.593679 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.594898 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.594902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.595018 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.595047 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 18:31:41.595121 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 18:31:41.595189 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":69,"thread_start_us":77,"threads_started":1}
I20260808 18:31:41.596000 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.590294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58848 (local address 127.7.164.65:33099)
0808 18:31:41.591002 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:41.591016 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:41.591052 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:31:41.591124 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:41.591129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:41.591297 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:41.591441 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:41.591701 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.592803 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.594093 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.594875 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.594989 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.595160 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:41.595726 (+ 566us) server_negotiation.cc:299] Negotiation successful
0808 18:31:41.595830 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":142,"threads_started":1}
I20260808 18:31:41.596001 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.589215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54782 (local address 127.7.164.66:39367)
0808 18:31:41.589695 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:41.589710 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:41.589735 (+ 25us) server_negotiation.cc:407] Connection header received
0808 18:31:41.589834 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:41.589839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:41.590058 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:41.590706 (+ 648us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:41.591002 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.592124 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.593802 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.594550 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.594644 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.594831 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:41.595420 (+ 589us) server_negotiation.cc:299] Negotiation successful
0808 18:31:41.595543 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":131,"threads_started":1}
I20260808 18:31:41.595628 4590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.590108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:33099 (local address 127.0.0.1:58848)
0808 18:31:41.590403 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:41.590439 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:41.590545 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:41.591445 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:41.591448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:41.591469 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:41.591573 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.591579 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.592935 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.592938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.593967 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.593975 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.595223 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.595227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.595316 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.595350 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 18:31:41.595449 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 18:31:41.595521 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":112,"threads_started":1}
I20260808 18:31:41.598918 4481 tablet_service.cc:1467] Processing CreateTablet for tablet 70feed3191fb498caf3267c577873bad (DEFAULT_TABLE table=test-table [id=a962b24926a44992867ed703b982f33b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:41.598883 4301 tablet_service.cc:1467] Processing CreateTablet for tablet 70feed3191fb498caf3267c577873bad (DEFAULT_TABLE table=test-table [id=a962b24926a44992867ed703b982f33b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:41.599947 4301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70feed3191fb498caf3267c577873bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:41.599993 4481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70feed3191fb498caf3267c577873bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:41.599833 4112 tablet_service.cc:1467] Processing CreateTablet for tablet 70feed3191fb498caf3267c577873bad (DEFAULT_TABLE table=test-table [id=a962b24926a44992867ed703b982f33b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:31:41.600759 4112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70feed3191fb498caf3267c577873bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:41.606467 4597 tablet_bootstrap.cc:492] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6: Bootstrap starting.
I20260808 18:31:41.608253 4599 tablet_bootstrap.cc:492] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144: Bootstrap starting.
I20260808 18:31:41.609018 4597 tablet_bootstrap.cc:654] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:41.609759 4597 log.cc:824] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:41.610069 4599 tablet_bootstrap.cc:654] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:41.610530 4600 tablet_bootstrap.cc:492] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17: Bootstrap starting.
I20260808 18:31:41.610895 4599 log.cc:824] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:41.611984 4597 tablet_bootstrap.cc:492] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6: No bootstrap required, opened a new log
I20260808 18:31:41.612215 4597 ts_tablet_manager.cc:1397] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:31:41.612952 4599 tablet_bootstrap.cc:492] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144: No bootstrap required, opened a new log
I20260808 18:31:41.613126 4600 tablet_bootstrap.cc:654] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:41.613166 4599 ts_tablet_manager.cc:1397] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:31:41.614045 4600 log.cc:824] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:41.616111 4600 tablet_bootstrap.cc:492] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17: No bootstrap required, opened a new log
I20260808 18:31:41.616005 4597 raft_consensus.cc:348] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d6d8944cbf429db731603bf2753ba6" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 39367 } } peers { permanent_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36305 } } peers { permanent_uuid: "0d59686e57a74ed7ad2f081bbfd38b17" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33099 } }
I20260808 18:31:41.616317 4600 ts_tablet_manager.cc:1397] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:31:41.616314 4597 raft_consensus.cc:374] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:41.616390 4597 raft_consensus.cc:729] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d6d8944cbf429db731603bf2753ba6, State: Initialized, Role: FOLLOWER
I20260808 18:31:41.616873 4597 consensus_queue.cc:260] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d6d8944cbf429db731603bf2753ba6" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 39367 } } peers { permanent_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36305 } } peers { permanent_uuid: "0d59686e57a74ed7ad2f081bbfd38b17" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33099 } }
I20260808 18:31:41.617594 4397 heartbeater.cc:503] Master 127.7.164.126:35307 was elected leader, sending a full tablet report...
I20260808 18:31:41.617873 4597 ts_tablet_manager.cc:1428] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 18:31:41.617523 4599 raft_consensus.cc:348] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d6d8944cbf429db731603bf2753ba6" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 39367 } } peers { permanent_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36305 } } peers { permanent_uuid: "0d59686e57a74ed7ad2f081bbfd38b17" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33099 } }
I20260808 18:31:41.618368 4599 raft_consensus.cc:374] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260808 18:31:41.618388 4398 tablet.cc:2369] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:31:41.618444 4599 raft_consensus.cc:729] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd80e1ed40a141dd85ee7444cbd8e144, State: Initialized, Role: FOLLOWER
I20260808 18:31:41.618857 4599 consensus_queue.cc:260] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d6d8944cbf429db731603bf2753ba6" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 39367 } } peers { permanent_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36305 } } peers { permanent_uuid: "0d59686e57a74ed7ad2f081bbfd38b17" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33099 } }
I20260808 18:31:41.619549 4579 heartbeater.cc:503] Master 127.7.164.126:35307 was elected leader, sending a full tablet report...
I20260808 18:31:41.619850 4599 ts_tablet_manager.cc:1428] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 18:31:41.623385 4600 raft_consensus.cc:348] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d6d8944cbf429db731603bf2753ba6" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 39367 } } peers { permanent_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36305 } } peers { permanent_uuid: "0d59686e57a74ed7ad2f081bbfd38b17" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33099 } }
I20260808 18:31:41.623595 4600 raft_consensus.cc:374] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:41.623629 4600 raft_consensus.cc:729] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d59686e57a74ed7ad2f081bbfd38b17, State: Initialized, Role: FOLLOWER
I20260808 18:31:41.623941 4600 consensus_queue.cc:260] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d6d8944cbf429db731603bf2753ba6" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 39367 } } peers { permanent_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36305 } } peers { permanent_uuid: "0d59686e57a74ed7ad2f081bbfd38b17" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33099 } }
I20260808 18:31:41.624565 4210 heartbeater.cc:503] Master 127.7.164.126:35307 was elected leader, sending a full tablet report...
I20260808 18:31:41.624663 4600 ts_tablet_manager.cc:1428] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
W20260808 18:31:41.682451 4211 tablet.cc:2369] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:31:41.800577 4606 raft_consensus.cc:482] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:31:41.800815 4606 raft_consensus.cc:504] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d6d8944cbf429db731603bf2753ba6" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 39367 } } peers { permanent_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36305 } } peers { permanent_uuid: "0d59686e57a74ed7ad2f081bbfd38b17" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33099 } }
I20260808 18:31:41.802026 4606 leader_election.cc:288] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65d6d8944cbf429db731603bf2753ba6 (127.7.164.66:39367), 0d59686e57a74ed7ad2f081bbfd38b17 (127.7.164.65:33099)
W20260808 18:31:41.802336 4580 tablet.cc:2369] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:31:41.806304 4581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.802320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.66:39367 (local address 127.7.164.67:49183)
0808 18:31:41.802649 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:41.802690 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:41.802776 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:41.803374 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:41.803377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:41.803395 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:41.803523 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.803550 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.804309 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.804312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.805105 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.805114 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.805798 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.805802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.805936 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.805977 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 18:31:41.806045 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 18:31:41.806138 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":8}
I20260808 18:31:41.806319 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.802926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:49183 (local address 127.7.164.66:39367)
0808 18:31:41.803124 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:41.803128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:41.803141 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:31:41.803187 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:41.803188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:41.803256 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:41.803337 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:41.803677 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.804156 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.805249 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.805657 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.805791 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.805844 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:41.806099 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 18:31:41.806171 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260808 18:31:41.806332 4608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.802328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:33099 (local address 127.7.164.67:38197)
0808 18:31:41.802855 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:41.802871 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:31:41.803009 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:31:41.803292 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:31:41.803298 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:31:41.803320 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:31:41.803479 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.803486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.804234 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.804240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.805116 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:31:41.805125 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.805785 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.805789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:31:41.805951 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.805990 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 18:31:41.806087 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 18:31:41.806161 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":141,"thread_start_us":135,"threads_started":1}
I20260808 18:31:41.806334 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.802411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.164.67:38197 (local address 127.7.164.65:33099)
0808 18:31:41.802562 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:41.802567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:41.802978 (+ 411us) server_negotiation.cc:407] Connection header received
0808 18:31:41.803108 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:41.803114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:41.803179 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:41.803252 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:31:41.803595 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.804117 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.805275 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.805656 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.805798 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.805849 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:41.806121 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 18:31:41.806197 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:31:41.807021 4139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70feed3191fb498caf3267c577873bad" candidate_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d59686e57a74ed7ad2f081bbfd38b17" is_pre_election: true
I20260808 18:31:41.807240 4332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70feed3191fb498caf3267c577873bad" candidate_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65d6d8944cbf429db731603bf2753ba6" is_pre_election: true
I20260808 18:31:41.807360 4139 raft_consensus.cc:2454] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd80e1ed40a141dd85ee7444cbd8e144 in term 0.
I20260808 18:31:41.807562 4332 raft_consensus.cc:2454] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd80e1ed40a141dd85ee7444cbd8e144 in term 0.
I20260808 18:31:41.807917 4418 leader_election.cc:302] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d59686e57a74ed7ad2f081bbfd38b17, fd80e1ed40a141dd85ee7444cbd8e144; no voters:
I20260808 18:31:41.808241 4606 raft_consensus.cc:2781] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:31:41.808317 4606 raft_consensus.cc:482] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:31:41.808348 4606 raft_consensus.cc:3037] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:41.809427 4606 raft_consensus.cc:504] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d6d8944cbf429db731603bf2753ba6" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 39367 } } peers { permanent_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36305 } } peers { permanent_uuid: "0d59686e57a74ed7ad2f081bbfd38b17" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33099 } }
I20260808 18:31:41.809813 4606 leader_election.cc:288] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [CANDIDATE]: Term 1 election: Requested vote from peers 65d6d8944cbf429db731603bf2753ba6 (127.7.164.66:39367), 0d59686e57a74ed7ad2f081bbfd38b17 (127.7.164.65:33099)
I20260808 18:31:41.810248 4332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70feed3191fb498caf3267c577873bad" candidate_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65d6d8944cbf429db731603bf2753ba6"
I20260808 18:31:41.810263 4139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70feed3191fb498caf3267c577873bad" candidate_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d59686e57a74ed7ad2f081bbfd38b17"
I20260808 18:31:41.810389 4332 raft_consensus.cc:3037] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:41.810389 4139 raft_consensus.cc:3037] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:41.811681 4139 raft_consensus.cc:2454] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd80e1ed40a141dd85ee7444cbd8e144 in term 1.
I20260808 18:31:41.811681 4332 raft_consensus.cc:2454] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd80e1ed40a141dd85ee7444cbd8e144 in term 1.
I20260808 18:31:41.811993 4418 leader_election.cc:302] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d59686e57a74ed7ad2f081bbfd38b17, fd80e1ed40a141dd85ee7444cbd8e144; no voters:
I20260808 18:31:41.812165 4606 raft_consensus.cc:2781] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:41.812387 4606 raft_consensus.cc:686] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [term 1 LEADER]: Becoming Leader. State: Replica: fd80e1ed40a141dd85ee7444cbd8e144, State: Running, Role: LEADER
I20260808 18:31:41.812742 4606 consensus_queue.cc:237] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d6d8944cbf429db731603bf2753ba6" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 39367 } } peers { permanent_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36305 } } peers { permanent_uuid: "0d59686e57a74ed7ad2f081bbfd38b17" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33099 } }
I20260808 18:31:41.816367 3943 catalog_manager.cc:5495] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd80e1ed40a141dd85ee7444cbd8e144 (127.7.164.67). New cstate: current_term: 1 leader_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65d6d8944cbf429db731603bf2753ba6" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd80e1ed40a141dd85ee7444cbd8e144" member_type: VOTER last_known_addr { host: "127.7.164.67" port: 36305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d59686e57a74ed7ad2f081bbfd38b17" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33099 } health_report { overall_health: UNKNOWN } } }
I20260808 18:31:41.830724 4594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.826769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45430 (local address 127.7.164.67:36305)
0808 18:31:41.826941 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:41.826945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:41.826963 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:31:41.827046 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:41.827048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:41.827106 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:41.827209 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:31:41.827470 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.827994 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.829206 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:41.829420 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:41.829520 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:41.829789 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:31:41.830202 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:31:41.830380 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:41.830484 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 18:31:41.830535 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 18:31:41.837203 4332 raft_consensus.cc:1261] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6 [term 1 FOLLOWER]: Refusing update from remote peer fd80e1ed40a141dd85ee7444cbd8e144: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:31:41.837203 4139 raft_consensus.cc:1261] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17 [term 1 FOLLOWER]: Refusing update from remote peer fd80e1ed40a141dd85ee7444cbd8e144: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:31:41.837971 4609 consensus_queue.cc:1035] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d59686e57a74ed7ad2f081bbfd38b17" member_type: VOTER last_known_addr { host: "127.7.164.65" port: 33099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:31:41.838186 4606 consensus_queue.cc:1035] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65d6d8944cbf429db731603bf2753ba6" member_type: VOTER last_known_addr { host: "127.7.164.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:31:41.851740 4623 mvcc.cc:205] Tried to move back new op lower bound from 7316332141916672000 to 7316332141827801088. Current Snapshot: MvccSnapshot[applied={T|T < 7316332141916672000}]
I20260808 18:31:41.854274 3939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:31:41.854446 3939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:31:41.857084 3939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a86f9e7694564446b754bfbcd9c44ec5: Sending DeleteTablet for 3 replicas of tablet 70feed3191fb498caf3267c577873bad
I20260808 18:31:41.857867 4481 tablet_service.cc:1514] Processing DeleteTablet for tablet 70feed3191fb498caf3267c577873bad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:41 UTC) from {username='slave'} at 127.0.0.1:45420
I20260808 18:31:41.858086 4112 tablet_service.cc:1514] Processing DeleteTablet for tablet 70feed3191fb498caf3267c577873bad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:41 UTC) from {username='slave'} at 127.0.0.1:58848
I20260808 18:31:41.858372 4301 tablet_service.cc:1514] Processing DeleteTablet for tablet 70feed3191fb498caf3267c577873bad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:31:41 UTC) from {username='slave'} at 127.0.0.1:54782
I20260808 18:31:41.858433 4632 tablet_replica.cc:330] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144: stopping tablet replica
I20260808 18:31:41.858757 4632 raft_consensus.cc:2229] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:31:41.858793 4634 tablet_replica.cc:330] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6: stopping tablet replica
I20260808 18:31:41.858938 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 4022
I20260808 18:31:41.859045 4633 tablet_replica.cc:330] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17: stopping tablet replica
I20260808 18:31:41.859097 4634 raft_consensus.cc:2229] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:31:41.859170 4632 raft_consensus.cc:2258] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:31:41.859306 4633 raft_consensus.cc:2229] T 70feed3191fb498caf3267c577873bad P 0d59686e57a74ed7ad2f081bbfd38b17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:31:41.859527 4634 raft_consensus.cc:2258] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:31:41.861049 4634 ts_tablet_manager.cc:1905] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:31:41.861173 4632 ts_tablet_manager.cc:1905] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:31:41.864071 4634 ts_tablet_manager.cc:1918] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:31:41.864243 4634 log.cc:1196] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-1/wal/wals/70feed3191fb498caf3267c577873bad
I20260808 18:31:41.864691 4634 ts_tablet_manager.cc:1939] T 70feed3191fb498caf3267c577873bad P 65d6d8944cbf429db731603bf2753ba6: Deleting consensus metadata
I20260808 18:31:41.864776 4632 ts_tablet_manager.cc:1918] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:31:41.864974 4632 log.cc:1196] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144: Deleting WAL directory at /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213640155572-7825-0/minicluster-data/ts-2/wal/wals/70feed3191fb498caf3267c577873bad
I20260808 18:31:41.865409 4632 ts_tablet_manager.cc:1939] T 70feed3191fb498caf3267c577873bad P fd80e1ed40a141dd85ee7444cbd8e144: Deleting consensus metadata
I20260808 18:31:41.866081 3920 catalog_manager.cc:4841] TS 65d6d8944cbf429db731603bf2753ba6 (127.7.164.66:39367): tablet 70feed3191fb498caf3267c577873bad (table test-table [id=a962b24926a44992867ed703b982f33b]) successfully deleted
I20260808 18:31:41.866811 3919 catalog_manager.cc:4841] TS fd80e1ed40a141dd85ee7444cbd8e144 (127.7.164.67:36305): tablet 70feed3191fb498caf3267c577873bad (table test-table [id=a962b24926a44992867ed703b982f33b]) successfully deleted
W20260808 18:31:41.871912 3917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.164.65:33099 (error 108)
I20260808 18:31:41.872848 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 4213
I20260808 18:31:41.873081 4590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:41.872744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.65:33099 (local address 127.0.0.1:58856)
0808 18:31:41.872881 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 18:31:41.872965 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.65:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 18:31:41.877745 3917 catalog_manager.cc:4584] TS 0d59686e57a74ed7ad2f081bbfd38b17 (127.7.164.65:33099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70feed3191fb498caf3267c577873bad: Network error: Client connection negotiation failed: client connection to 127.7.164.65:33099: connect: Connection refused (error 111)
I20260808 18:31:41.889689 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 4401
I20260808 18:31:41.900449 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 3894
2026-08-08T18:31:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1193 ms)
[----------] 6 tests from EncryptionPolicyTest (11083 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:41 dist-test-slave-jztv krb5kdc[4642](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:41 dist-test-slave-jztv krb5kdc[4642](info): set up 2 sockets
Aug 08 18:31:41 dist-test-slave-jztv krb5kdc[4642](info): commencing operation
krb5kdc: starting...
W20260808 18:31:43.121807 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:31:43 dist-test-slave-jztv krb5kdc[4642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:31:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:31:43.257956 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:43.372793 4665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:43.373126 4665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:43.373212 4665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:43.377347 4665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:43.377453 4665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:43.377499 4665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:43.377557 4665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:43.377599 4665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:43.383077 4665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32777
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:43.384212 4665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:43.385260 4665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:43.393939 4676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:43.394032 4674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:43.393913 4673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:43.394744 4665 server_base.cc:1034] running on GCE node
I20260808 18:31:43.395336 4665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:43.396463 4665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:43.397699 4665 hybrid_clock.cc:648] HybridClock initialized: now 1786213903397668 us; error 40 us; skew 500 ppm
Aug 08 18:31:43 dist-test-slave-jztv krb5kdc[4642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213903, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:43.400604 4665 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:31:43.401595 4665 webserver.cc:460] Webserver started at http://127.0.0.1:41643/ using document root <none> and password file <none>
I20260808 18:31:43.402200 4665 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:43.402354 4665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:43.402608 4665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:43.404780 4665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "8e8b183eeb3f4f4f8b1f4136448e252e"
format_stamp: "Formatted at 2026-08-08 18:31:43 on dist-test-slave-jztv"
server_key: "594f61f814ee0b1681dceaa051e1961d"
server_key_iv: "3acf3fbaf99214b8a6c56666ec1926d3"
server_key_version: "encryptionkey@0"
I20260808 18:31:43.405422 4665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "8e8b183eeb3f4f4f8b1f4136448e252e"
format_stamp: "Formatted at 2026-08-08 18:31:43 on dist-test-slave-jztv"
server_key: "594f61f814ee0b1681dceaa051e1961d"
server_key_iv: "3acf3fbaf99214b8a6c56666ec1926d3"
server_key_version: "encryptionkey@0"
I20260808 18:31:43.409332 4665 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 18:31:43.411938 4687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:43.413213 4665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 18:31:43.413365 4665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "8e8b183eeb3f4f4f8b1f4136448e252e"
format_stamp: "Formatted at 2026-08-08 18:31:43 on dist-test-slave-jztv"
server_key: "594f61f814ee0b1681dceaa051e1961d"
server_key_iv: "3acf3fbaf99214b8a6c56666ec1926d3"
server_key_version: "encryptionkey@0"
I20260808 18:31:43.413494 4665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:43.428510 4665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:43.431471 4665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:43.431689 4665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:43.439149 4665 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32777
I20260808 18:31:43.439178 4754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32777 every 8 connection(s)
I20260808 18:31:43.440136 4665 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:43.443020 4755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:43.445664 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 4665
I20260808 18:31:43.445777 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:43.446110 7825 external_mini_cluster.cc:1428] Setting key 73654bd23ec4213cabf6c08a7bcbbc37
I20260808 18:31:43.449563 4755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e: Bootstrap starting.
I20260808 18:31:43.452374 4755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:43.453217 4755 log.cc:824] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:43.455652 4755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e: No bootstrap required, opened a new log
Aug 08 18:31:43 dist-test-slave-jztv krb5kdc[4642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 18:31:43.459421 4755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e8b183eeb3f4f4f8b1f4136448e252e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32777 } }
I20260808 18:31:43.459736 4755 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:43.459841 4755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e8b183eeb3f4f4f8b1f4136448e252e, State: Initialized, Role: FOLLOWER
I20260808 18:31:43.460408 4755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e8b183eeb3f4f4f8b1f4136448e252e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32777 } }
I20260808 18:31:43.460548 4755 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:43.460651 4755 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:43.460776 4755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:43.461926 4755 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e8b183eeb3f4f4f8b1f4136448e252e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32777 } }
I20260808 18:31:43.462343 4755 leader_election.cc:302] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e8b183eeb3f4f4f8b1f4136448e252e; no voters:
I20260808 18:31:43.462635 4758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:43.447670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45611 (local address 127.0.0.1:32777)
0808 18:31:43.448152 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:43.448172 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:43.448222 (+ 50us) server_negotiation.cc:407] Connection header received
0808 18:31:43.449077 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:43.449101 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:43.449533 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:43.449907 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:43.450698 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:43.452218 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:43.453134 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:43.453799 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:43.453931 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:43.456875 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:43.456880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:43.456893 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:43.456926 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:43.459212 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:43.459820 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:43.459827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:43.459833 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:43.460009 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:43.460361 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:43.460364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:43.460365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:43.460928 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:43.461104 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:43.461452 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 18:31:43.461718 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":144,"threads_started":1}
I20260808 18:31:43.462704 4755 leader_election.cc:288] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:43.462783 4760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:43.463012 4760 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [term 1 LEADER]: Becoming Leader. State: Replica: 8e8b183eeb3f4f4f8b1f4136448e252e, State: Running, Role: LEADER
I20260808 18:31:43.463387 4760 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e8b183eeb3f4f4f8b1f4136448e252e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32777 } }
I20260808 18:31:43.464023 4755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:43.465091 4762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e8b183eeb3f4f4f8b1f4136448e252e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e8b183eeb3f4f4f8b1f4136448e252e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32777 } } }
I20260808 18:31:43.465229 4762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:43.465643 4774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:43.466442 4763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e8b183eeb3f4f4f8b1f4136448e252e. Latest consensus state: current_term: 1 leader_uuid: "8e8b183eeb3f4f4f8b1f4136448e252e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e8b183eeb3f4f4f8b1f4136448e252e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32777 } } }
I20260808 18:31:43.466542 4763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:43.467002 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:31:43.469194 4774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:43.475498 4774 catalog_manager.cc:1347] Generated new cluster ID: ae7153d5c3cc45918f39e93cbfb73de1
I20260808 18:31:43.475641 4774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:43.480847 4758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:43.470685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40145 (local address 127.0.0.1:32777)
0808 18:31:43.470787 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:43.470791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:43.471206 (+ 415us) server_negotiation.cc:407] Connection header received
0808 18:31:43.471355 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:43.471359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:43.471410 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:43.471502 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:43.472353 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:43.473182 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:43.474473 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:43.474844 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:43.474936 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:43.477004 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:43.477008 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:43.477011 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:43.477040 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:43.478854 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:43.479438 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:43.479442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:43.479444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:43.479534 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:43.479905 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:43.479908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:43.479910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:43.480120 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:43.480272 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:43.480582 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 18:31:43.480684 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 18:31:43.495090 4774 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:43.496430 4774 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:43.503969 4774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e8b183eeb3f4f4f8b1f4136448e252e: Generated new TSK 0
I20260808 18:31:43.504753 4774 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:31:43.514302 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 4665
W20260808 18:31:43.533048 4638 connection.cc:535] client connection to 127.0.0.1:32777 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32777): Connection reset by peer (error 104)
2026-08-08T18:31:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1619 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:43 dist-test-slave-jztv krb5kdc[4800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:43 dist-test-slave-jztv krb5kdc[4800](info): set up 2 sockets
Aug 08 18:31:43 dist-test-slave-jztv krb5kdc[4800](info): commencing operation
krb5kdc: starting...
W20260808 18:31:44.801002 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:31:44 dist-test-slave-jztv krb5kdc[4800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:31:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:31:44.937834 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44661
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:45.056074 4818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:45.056447 4818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:45.056533 4818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:45.060822 4818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:45.060945 4818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:45.060984 4818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:45.061019 4818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:45.061049 4818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:45.066773 4818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44661
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:45.068017 4818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:45.069111 4818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:45.078079 4825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:45.078109 4831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:45.078112 4824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:45.078477 4818 server_base.cc:1034] running on GCE node
I20260808 18:31:45.079208 4818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:45.080348 4818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:45.081560 4818 hybrid_clock.cc:648] HybridClock initialized: now 1786213905081527 us; error 34 us; skew 500 ppm
Aug 08 18:31:45 dist-test-slave-jztv krb5kdc[4800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213905, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:45.088568 4818 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:31:45.089635 4818 webserver.cc:460] Webserver started at http://127.0.0.1:42387/ using document root <none> and password file <none>
I20260808 18:31:45.090201 4818 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:45.090348 4818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:45.090545 4818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:45.092988 4818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "0171f98dd9f941ab9e8a9aa583003e46"
format_stamp: "Formatted at 2026-08-08 18:31:45 on dist-test-slave-jztv"
server_key: "c50b59474be16b43e55ff0b7442816f1"
server_key_iv: "a11a1675e7e53705170cea36e5eaf4ea"
server_key_version: "encryptionkey@0"
I20260808 18:31:45.093571 4818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "0171f98dd9f941ab9e8a9aa583003e46"
format_stamp: "Formatted at 2026-08-08 18:31:45 on dist-test-slave-jztv"
server_key: "c50b59474be16b43e55ff0b7442816f1"
server_key_iv: "a11a1675e7e53705170cea36e5eaf4ea"
server_key_version: "encryptionkey@0"
I20260808 18:31:45.097612 4818 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 18:31:45.100160 4841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:45.101425 4818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:31:45.101577 4818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "0171f98dd9f941ab9e8a9aa583003e46"
format_stamp: "Formatted at 2026-08-08 18:31:45 on dist-test-slave-jztv"
server_key: "c50b59474be16b43e55ff0b7442816f1"
server_key_iv: "a11a1675e7e53705170cea36e5eaf4ea"
server_key_version: "encryptionkey@0"
I20260808 18:31:45.101702 4818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:45.127681 4818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:45.131137 4818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:45.131369 4818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:45.139441 4818 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44661
I20260808 18:31:45.139446 4919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44661 every 8 connection(s)
I20260808 18:31:45.140547 4818 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:45.143546 4921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:45.145798 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 4818
I20260808 18:31:45.145948 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:45.146333 7825 external_mini_cluster.cc:1428] Setting key ef21736d61cb4169cf75da9d6e023cdb
I20260808 18:31:45.150319 4921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46: Bootstrap starting.
I20260808 18:31:45.152673 4921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:45.153484 4921 log.cc:824] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:45.155884 4921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46: No bootstrap required, opened a new log
I20260808 18:31:45.156364 4926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:45.148185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55795 (local address 127.0.0.1:44661)
0808 18:31:45.148660 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:45.148682 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:45.148732 (+ 50us) server_negotiation.cc:407] Connection header received
0808 18:31:45.149412 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:45.149430 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:45.149794 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:45.150158 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:45.150642 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:45.151629 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:45.152578 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:45.153136 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:45.153324 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:45.154382 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:45.154399 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:45.154419 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:45.154826 (+ 407us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:45.154884 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:45.154912 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:45.155023 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:45.155373 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 18:31:45.155553 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":170,"threads_started":1}
I20260808 18:31:45.159976 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:31:45.160467 4921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0171f98dd9f941ab9e8a9aa583003e46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44661 } }
I20260808 18:31:45.160748 4921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:45.160836 4921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0171f98dd9f941ab9e8a9aa583003e46, State: Initialized, Role: FOLLOWER
I20260808 18:31:45.161305 4921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0171f98dd9f941ab9e8a9aa583003e46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44661 } }
I20260808 18:31:45.161499 4921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:45.161576 4921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:45.161705 4921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:45.163105 4921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0171f98dd9f941ab9e8a9aa583003e46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44661 } }
I20260808 18:31:45.163522 4921 leader_election.cc:302] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0171f98dd9f941ab9e8a9aa583003e46; no voters:
I20260808 18:31:45.163894 4921 leader_election.cc:288] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:45.164059 4940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:45.164386 4940 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [term 1 LEADER]: Becoming Leader. State: Replica: 0171f98dd9f941ab9e8a9aa583003e46, State: Running, Role: LEADER
I20260808 18:31:45.164770 4940 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0171f98dd9f941ab9e8a9aa583003e46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44661 } }
I20260808 18:31:45.165128 4921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:45.166695 4941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0171f98dd9f941ab9e8a9aa583003e46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0171f98dd9f941ab9e8a9aa583003e46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44661 } } }
I20260808 18:31:45.166915 4941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:45.167217 4941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0171f98dd9f941ab9e8a9aa583003e46. Latest consensus state: current_term: 1 leader_uuid: "0171f98dd9f941ab9e8a9aa583003e46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0171f98dd9f941ab9e8a9aa583003e46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44661 } } }
I20260808 18:31:45.167560 4926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:45.162340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51191 (local address 127.0.0.1:44661)
0808 18:31:45.162480 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:45.162485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:45.162646 (+ 161us) server_negotiation.cc:407] Connection header received
0808 18:31:45.162734 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:45.162737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:45.162792 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:45.162891 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:45.163141 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:45.163995 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:45.165239 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:45.165617 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:45.165729 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:45.166819 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:45.166823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:45.166826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:45.166843 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:45.166876 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:45.166891 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:45.166984 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:45.167375 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 18:31:45.167415 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:31:45.167728 4941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46 [sys.catalog]: This master's current role is: LEADER
W20260808 18:31:45.170161 4957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:31:45.170298 4957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:31:45.170411 4955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:45.171124 4955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:45.176754 4955 catalog_manager.cc:1347] Generated new cluster ID: 517fb0888a854daf9b767d415e77f096
I20260808 18:31:45.176836 4955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:45.185571 4955 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:45.186836 4955 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:45.194566 4955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0171f98dd9f941ab9e8a9aa583003e46: Generated new TSK 0
I20260808 18:31:45.195401 4955 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:31:45.205298 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 4818
W20260808 18:31:45.220920 4795 connection.cc:535] client connection to 127.0.0.1:44661 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44661): Connection reset by peer (error 104)
2026-08-08T18:31:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1687 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:45 dist-test-slave-jztv krb5kdc[4966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:45 dist-test-slave-jztv krb5kdc[4966](info): set up 2 sockets
Aug 08 18:31:45 dist-test-slave-jztv krb5kdc[4966](info): commencing operation
krb5kdc: starting...
W20260808 18:31:46.436321 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:31:46 dist-test-slave-jztv krb5kdc[4966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:31:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:31:46.571527 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39911
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:46.689203 4984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:46.689553 4984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:46.689639 4984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:46.693667 4984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:46.693785 4984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:46.693823 4984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:46.693861 4984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:46.693892 4984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:46.699254 4984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39911
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:46.700376 4984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:46.701427 4984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:46.710100 4995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:46.710180 4996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:46.710246 4984 server_base.cc:1034] running on GCE node
W20260808 18:31:46.710105 4998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:46.711046 4984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:46.712217 4984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:46.713492 4984 hybrid_clock.cc:648] HybridClock initialized: now 1786213906713469 us; error 36 us; skew 500 ppm
Aug 08 18:31:46 dist-test-slave-jztv krb5kdc[4966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213906, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:46.716528 4984 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:31:46.717424 4984 webserver.cc:460] Webserver started at http://127.0.0.1:43281/ using document root <none> and password file <none>
I20260808 18:31:46.717963 4984 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:46.718024 4984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:46.718195 4984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:46.720395 4984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "73820f949ff04c8d9fdeff6b724d4034"
format_stamp: "Formatted at 2026-08-08 18:31:46 on dist-test-slave-jztv"
server_key: "839dd07640300611d5322d75f0b767a8"
server_key_iv: "597d63068a486d397a5282190d715033"
server_key_version: "encryptionkey@0"
I20260808 18:31:46.720927 4984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "73820f949ff04c8d9fdeff6b724d4034"
format_stamp: "Formatted at 2026-08-08 18:31:46 on dist-test-slave-jztv"
server_key: "839dd07640300611d5322d75f0b767a8"
server_key_iv: "597d63068a486d397a5282190d715033"
server_key_version: "encryptionkey@0"
I20260808 18:31:46.724645 4984 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 18:31:46.727128 5006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:46.728344 4984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:31:46.728467 4984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "73820f949ff04c8d9fdeff6b724d4034"
format_stamp: "Formatted at 2026-08-08 18:31:46 on dist-test-slave-jztv"
server_key: "839dd07640300611d5322d75f0b767a8"
server_key_iv: "597d63068a486d397a5282190d715033"
server_key_version: "encryptionkey@0"
I20260808 18:31:46.728559 4984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:46.733099 4984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:46.735970 4984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:46.736114 4984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:46.742924 4984 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39911
I20260808 18:31:46.742993 5089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39911 every 8 connection(s)
I20260808 18:31:46.743884 4984 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:46.746703 5091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:46.749311 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 4984
I20260808 18:31:46.749477 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:46.749763 7825 external_mini_cluster.cc:1428] Setting key a9b7fa5c6a1a2c3bff18075fda9d4d82
I20260808 18:31:46.752894 5091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034: Bootstrap starting.
I20260808 18:31:46.755369 5098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:46.751167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33853 (local address 127.0.0.1:39911)
0808 18:31:46.751683 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:46.751702 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:46.751745 (+ 43us) server_negotiation.cc:407] Connection header received
0808 18:31:46.752556 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:46.752578 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:46.752920 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:46.753282 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:46.753982 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:46.753990 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:46.754017 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:46.754032 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:46.754101 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:46.754133 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:46.754309 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:46.754386 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 18:31:46.754530 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":167,"threads_started":1}
I20260808 18:31:46.755646 5091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:46.756527 5091 log.cc:824] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:46.758315 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:31:46.758746 5091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034: No bootstrap required, opened a new log
I20260808 18:31:46.762609 5091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73820f949ff04c8d9fdeff6b724d4034" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39911 } }
I20260808 18:31:46.762853 5091 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:46.762914 5091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73820f949ff04c8d9fdeff6b724d4034, State: Initialized, Role: FOLLOWER
I20260808 18:31:46.763315 5091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73820f949ff04c8d9fdeff6b724d4034" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39911 } }
I20260808 18:31:46.763433 5091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:46.763527 5091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:46.763641 5091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:46.764060 5098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:46.760321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42611 (local address 127.0.0.1:39911)
0808 18:31:46.760437 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:46.760442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:46.762505 (+ 2063us) server_negotiation.cc:407] Connection header received
0808 18:31:46.762663 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:46.762671 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:46.762702 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:46.762786 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:46.763644 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:46.763647 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:46.763650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:46.763669 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:46.763712 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:46.763728 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:46.763812 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:46.763890 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 18:31:46.763931 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:31:46.764739 5091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73820f949ff04c8d9fdeff6b724d4034" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39911 } }
I20260808 18:31:46.765095 5091 leader_election.cc:302] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73820f949ff04c8d9fdeff6b724d4034; no voters:
I20260808 18:31:46.765410 5091 leader_election.cc:288] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:46.765534 5107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:46.765774 5107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [term 1 LEADER]: Becoming Leader. State: Replica: 73820f949ff04c8d9fdeff6b724d4034, State: Running, Role: LEADER
I20260808 18:31:46.766129 5107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73820f949ff04c8d9fdeff6b724d4034" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39911 } }
I20260808 18:31:46.766753 5091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:46.767880 5109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73820f949ff04c8d9fdeff6b724d4034. Latest consensus state: current_term: 1 leader_uuid: "73820f949ff04c8d9fdeff6b724d4034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73820f949ff04c8d9fdeff6b724d4034" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39911 } } }
I20260808 18:31:46.768023 5109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:46.768150 5108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73820f949ff04c8d9fdeff6b724d4034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73820f949ff04c8d9fdeff6b724d4034" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39911 } } }
I20260808 18:31:46.768288 5108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:46.769129 5126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:46.771661 5126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:46.776873 5126 catalog_manager.cc:1347] Generated new cluster ID: 41d48665ef224cae9a66e87eb0cdc679
I20260808 18:31:46.776940 5126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:46.795511 5126 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:46.795886 5126 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:46.802403 5126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73820f949ff04c8d9fdeff6b724d4034: Generated new TSK 0
I20260808 18:31:46.803110 5126 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:31:46.812510 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 4984
2026-08-08T18:31:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1605 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:46 dist-test-slave-jztv krb5kdc[5144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:46 dist-test-slave-jztv krb5kdc[5144](info): set up 2 sockets
Aug 08 18:31:46 dist-test-slave-jztv krb5kdc[5144](info): commencing operation
krb5kdc: starting...
W20260808 18:31:48.055269 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:31:48 dist-test-slave-jztv krb5kdc[5144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:31:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:31:48.187533 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33013
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:48.304785 5161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:48.305075 5161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:48.305119 5161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:48.309325 5161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:48.309402 5161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:48.309422 5161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:48.309439 5161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:48.309455 5161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:48.315088 5161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:48.316329 5161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:48.317409 5161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:48.326195 5172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:48.326226 5170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:48.326465 5161 server_base.cc:1034] running on GCE node
W20260808 18:31:48.326263 5174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:48.327316 5161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:48.328531 5161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:48.329752 5161 hybrid_clock.cc:648] HybridClock initialized: now 1786213908329732 us; error 34 us; skew 500 ppm
Aug 08 18:31:48 dist-test-slave-jztv krb5kdc[5144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213908, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:48.332862 5161 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:31:48.333935 5161 webserver.cc:460] Webserver started at http://127.0.0.1:44371/ using document root <none> and password file <none>
I20260808 18:31:48.334599 5161 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:48.334666 5161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:48.334841 5161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:48.336935 5161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "c5daf95db9c74a3ca0007e203f8c7d9e"
format_stamp: "Formatted at 2026-08-08 18:31:48 on dist-test-slave-jztv"
server_key: "bf59cb81de24a1bc2f35756fedae7eeb"
server_key_iv: "c01a70cbee5a072dfd3ffcc8d222b979"
server_key_version: "encryptionkey@0"
I20260808 18:31:48.337491 5161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "c5daf95db9c74a3ca0007e203f8c7d9e"
format_stamp: "Formatted at 2026-08-08 18:31:48 on dist-test-slave-jztv"
server_key: "bf59cb81de24a1bc2f35756fedae7eeb"
server_key_iv: "c01a70cbee5a072dfd3ffcc8d222b979"
server_key_version: "encryptionkey@0"
I20260808 18:31:48.341836 5161 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 18:31:48.344734 5183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:48.346192 5161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:31:48.346451 5161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "c5daf95db9c74a3ca0007e203f8c7d9e"
format_stamp: "Formatted at 2026-08-08 18:31:48 on dist-test-slave-jztv"
server_key: "bf59cb81de24a1bc2f35756fedae7eeb"
server_key_iv: "c01a70cbee5a072dfd3ffcc8d222b979"
server_key_version: "encryptionkey@0"
I20260808 18:31:48.346679 5161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:48.368484 5161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:48.371953 5161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:48.372198 5161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:48.380125 5161 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33013
I20260808 18:31:48.380137 5273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33013 every 8 connection(s)
I20260808 18:31:48.381187 5161 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:48.384230 5274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:48.385073 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 5161
I20260808 18:31:48.385247 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:48.385537 7825 external_mini_cluster.cc:1428] Setting key 9573e1abf40e8b96051f5f45c78454c1
I20260808 18:31:48.390731 5274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e: Bootstrap starting.
I20260808 18:31:48.393502 5274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:48.394526 5274 log.cc:824] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:31:48 dist-test-slave-jztv krb5kdc[5144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 18:31:48.396684 5274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e: No bootstrap required, opened a new log
I20260808 18:31:48.400830 5274 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5daf95db9c74a3ca0007e203f8c7d9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33013 } }
I20260808 18:31:48.401077 5280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:48.387061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53317 (local address 127.0.0.1:33013)
0808 18:31:48.387495 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:48.387512 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:48.387549 (+ 37us) server_negotiation.cc:407] Connection header received
0808 18:31:48.388181 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:48.388199 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:48.388541 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:48.388816 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:48.389771 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:48.390894 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:48.392079 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:48.392571 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:48.392691 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:48.395851 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:48.395858 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:48.395872 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:48.395899 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:48.398017 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:48.398610 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:48.398615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:48.398620 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:48.398738 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:48.399074 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:48.399076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:48.399077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:48.399496 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:48.399657 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:48.399978 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 18:31:48.400331 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":130,"threads_started":1}
I20260808 18:31:48.401093 5274 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:48.401261 5274 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5daf95db9c74a3ca0007e203f8c7d9e, State: Initialized, Role: FOLLOWER
I20260808 18:31:48.401780 5274 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5daf95db9c74a3ca0007e203f8c7d9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33013 } }
I20260808 18:31:48.401957 5274 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:48.402024 5274 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:48.402110 5274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:48.403126 5274 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5daf95db9c74a3ca0007e203f8c7d9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33013 } }
I20260808 18:31:48.403489 5274 leader_election.cc:302] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5daf95db9c74a3ca0007e203f8c7d9e; no voters:
I20260808 18:31:48.403872 5274 leader_election.cc:288] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:48.403949 5283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:48.404237 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:31:48.404263 5283 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [term 1 LEADER]: Becoming Leader. State: Replica: c5daf95db9c74a3ca0007e203f8c7d9e, State: Running, Role: LEADER
I20260808 18:31:48.404578 5283 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5daf95db9c74a3ca0007e203f8c7d9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33013 } }
I20260808 18:31:48.405185 5274 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:48.405787 5283 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5daf95db9c74a3ca0007e203f8c7d9e. Latest consensus state: current_term: 1 leader_uuid: "c5daf95db9c74a3ca0007e203f8c7d9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5daf95db9c74a3ca0007e203f8c7d9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33013 } } }
I20260808 18:31:48.405912 5283 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:48.406255 5284 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5daf95db9c74a3ca0007e203f8c7d9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5daf95db9c74a3ca0007e203f8c7d9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33013 } } }
I20260808 18:31:48.406374 5284 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:48.406908 5297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:48.410244 5297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:48.417138 5297 catalog_manager.cc:1347] Generated new cluster ID: 19e40d5bafd94ca9a7008bc844861028
I20260808 18:31:48.417256 5297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:48.417912 5280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:48.406787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39607 (local address 127.0.0.1:33013)
0808 18:31:48.406888 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:48.406892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:48.408772 (+ 1880us) server_negotiation.cc:407] Connection header received
0808 18:31:48.408959 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:48.408988 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:48.409037 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:48.409090 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:48.410236 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:48.411012 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:48.412119 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:48.412494 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:48.412607 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:48.414301 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:48.414305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:48.414308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:48.414333 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:48.415874 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:48.416360 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:48.416363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:48.416365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:48.416451 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:48.416773 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:48.416776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:48.416778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:48.416996 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:48.417117 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:48.417518 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 18:31:48.417635 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 18:31:48.434196 5297 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:48.435161 5297 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:48.440021 5297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5daf95db9c74a3ca0007e203f8c7d9e: Generated new TSK 0
I20260808 18:31:48.440755 5297 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:31:48.449649 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 5161
2026-08-08T18:31:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1638 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:48 dist-test-slave-jztv krb5kdc[5319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:48 dist-test-slave-jztv krb5kdc[5319](info): set up 2 sockets
Aug 08 18:31:48 dist-test-slave-jztv krb5kdc[5319](info): commencing operation
krb5kdc: starting...
W20260808 18:31:49.668414 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:31:49 dist-test-slave-jztv krb5kdc[5319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:31:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:31:49.801852 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39725
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:49.918458 5344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:49.918763 5344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:49.918843 5344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:49.922935 5344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:49.923026 5344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:49.923063 5344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:49.923125 5344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:49.923166 5344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:49.928543 5344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:49.929646 5344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:49.930750 5344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:49.939683 5353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:49.939721 5355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:49.939721 5351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:49.940016 5344 server_base.cc:1034] running on GCE node
I20260808 18:31:49.940585 5344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:49.941756 5344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:49.942957 5344 hybrid_clock.cc:648] HybridClock initialized: now 1786213909942936 us; error 33 us; skew 500 ppm
Aug 08 18:31:49 dist-test-slave-jztv krb5kdc[5319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213909, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:49.945986 5344 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:31:49.947053 5344 webserver.cc:460] Webserver started at http://127.0.0.1:44075/ using document root <none> and password file <none>
I20260808 18:31:49.947678 5344 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:49.947780 5344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:49.948035 5344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:49.950356 5344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "6fea4ec814144fa0ada4397e87b80939"
format_stamp: "Formatted at 2026-08-08 18:31:49 on dist-test-slave-jztv"
server_key: "b18a9de7bddca3dd5c962d50b9430769"
server_key_iv: "f7a2ffd1d3a5c6ae59550f6b088e0a16"
server_key_version: "encryptionkey@0"
I20260808 18:31:49.950954 5344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "6fea4ec814144fa0ada4397e87b80939"
format_stamp: "Formatted at 2026-08-08 18:31:49 on dist-test-slave-jztv"
server_key: "b18a9de7bddca3dd5c962d50b9430769"
server_key_iv: "f7a2ffd1d3a5c6ae59550f6b088e0a16"
server_key_version: "encryptionkey@0"
I20260808 18:31:49.954813 5344 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:31:49.957365 5364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:49.958771 5344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:31:49.958925 5344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "6fea4ec814144fa0ada4397e87b80939"
format_stamp: "Formatted at 2026-08-08 18:31:49 on dist-test-slave-jztv"
server_key: "b18a9de7bddca3dd5c962d50b9430769"
server_key_iv: "f7a2ffd1d3a5c6ae59550f6b088e0a16"
server_key_version: "encryptionkey@0"
I20260808 18:31:49.959064 5344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:49.970050 5344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:49.973676 5344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:49.973956 5344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:49.981997 5344 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39725
I20260808 18:31:49.982007 5451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39725 every 8 connection(s)
I20260808 18:31:49.983062 5344 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:49.986017 5455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:49.990176 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 5344
I20260808 18:31:49.990363 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:49.990597 7825 external_mini_cluster.cc:1428] Setting key 9ba0b7cd97f689f776bc077a93692d43
I20260808 18:31:49.993063 5455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939: Bootstrap starting.
I20260808 18:31:49.995391 5455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:49.996174 5455 log.cc:824] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:49.998292 5455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939: No bootstrap required, opened a new log
I20260808 18:31:50.000372 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:49.992026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32899 (local address 127.0.0.1:39725)
0808 18:31:49.992459 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:49.992481 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:49.992708 (+ 227us) server_negotiation.cc:407] Connection header received
0808 18:31:49.993556 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:49.993575 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:49.993927 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:49.994288 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:49.994713 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:49.995760 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:49.996743 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:49.997240 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:49.997425 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:49.998508 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:49.998513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:49.998526 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:49.998925 (+ 399us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:49.998979 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:49.999002 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:49.999120 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:49.999236 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 18:31:49.999461 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":138,"threads_started":1}
I20260808 18:31:50.001610 5455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fea4ec814144fa0ada4397e87b80939" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39725 } }
I20260808 18:31:50.001886 5455 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:50.001937 5455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fea4ec814144fa0ada4397e87b80939, State: Initialized, Role: FOLLOWER
I20260808 18:31:50.002455 5455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fea4ec814144fa0ada4397e87b80939" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39725 } }
I20260808 18:31:50.002631 5455 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:50.002707 5455 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:50.002815 5455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:50.003553 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:31:50.004093 5455 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fea4ec814144fa0ada4397e87b80939" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39725 } }
I20260808 18:31:50.004489 5455 leader_election.cc:302] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fea4ec814144fa0ada4397e87b80939; no voters:
I20260808 18:31:50.004866 5455 leader_election.cc:288] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:50.004987 5463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:50.005291 5463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [term 1 LEADER]: Becoming Leader. State: Replica: 6fea4ec814144fa0ada4397e87b80939, State: Running, Role: LEADER
I20260808 18:31:50.005715 5463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fea4ec814144fa0ada4397e87b80939" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39725 } }
I20260808 18:31:50.006347 5455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:50.007134 5463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fea4ec814144fa0ada4397e87b80939. Latest consensus state: current_term: 1 leader_uuid: "6fea4ec814144fa0ada4397e87b80939" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fea4ec814144fa0ada4397e87b80939" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39725 } } }
I20260808 18:31:50.007330 5463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:50.007614 5471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fea4ec814144fa0ada4397e87b80939" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fea4ec814144fa0ada4397e87b80939" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39725 } } }
I20260808 18:31:50.007728 5471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:50.007835 5483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:50.011152 5483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:50.011369 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:50.005959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48633 (local address 127.0.0.1:39725)
0808 18:31:50.006256 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:50.006260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:50.006317 (+ 57us) server_negotiation.cc:407] Connection header received
0808 18:31:50.006548 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:50.006551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:50.006602 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:50.006702 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:50.007033 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:50.007856 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:50.009303 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:50.009664 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:50.009744 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:50.010861 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:50.010865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:50.010868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:50.010883 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:50.010921 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:50.010935 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:50.011019 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:50.011144 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 18:31:50.011239 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260808 18:31:50.017181 5483 catalog_manager.cc:1347] Generated new cluster ID: d46f696cae454d72b9bcf902ca520b6e
I20260808 18:31:50.017266 5483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:50.025341 5483 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:50.026362 5483 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:50.034381 5483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fea4ec814144fa0ada4397e87b80939: Generated new TSK 0
I20260808 18:31:50.035172 5483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:31:50.038543 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 5344
2026-08-08T18:31:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1584 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:50 dist-test-slave-jztv krb5kdc[5499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:50 dist-test-slave-jztv krb5kdc[5499](info): set up 2 sockets
Aug 08 18:31:50 dist-test-slave-jztv krb5kdc[5499](info): commencing operation
krb5kdc: starting...
W20260808 18:31:51.279422 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:31:51 dist-test-slave-jztv krb5kdc[5499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:31:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:31:51.413197 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41025
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:51.529136 5520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:51.529438 5520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:51.529525 5520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:51.533547 5520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:51.533643 5520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:51.533689 5520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:51.533739 5520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:51.533775 5520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:51.539021 5520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:51.540062 5520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:51.541131 5520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:31:51.550256 5520 server_base.cc:1034] running on GCE node
W20260808 18:31:51.550271 5526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:51.550279 5529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:51.550210 5525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:51.551203 5520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:51.552412 5520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:51.553654 5520 hybrid_clock.cc:648] HybridClock initialized: now 1786213911553632 us; error 33 us; skew 500 ppm
Aug 08 18:31:51 dist-test-slave-jztv krb5kdc[5499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213911, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:51.556602 5520 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:31:51.557509 5520 webserver.cc:460] Webserver started at http://127.0.0.1:38187/ using document root <none> and password file <none>
I20260808 18:31:51.558081 5520 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:51.558182 5520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:51.558454 5520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:51.560576 5520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "9b2d6ac756f54a959363750fb11e6b47"
format_stamp: "Formatted at 2026-08-08 18:31:51 on dist-test-slave-jztv"
server_key: "c6d4c47cc781e7c83261827c12dc42b7"
server_key_iv: "88618c2f0426c320e407a5c158170b72"
server_key_version: "encryptionkey@0"
I20260808 18:31:51.561160 5520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "9b2d6ac756f54a959363750fb11e6b47"
format_stamp: "Formatted at 2026-08-08 18:31:51 on dist-test-slave-jztv"
server_key: "c6d4c47cc781e7c83261827c12dc42b7"
server_key_iv: "88618c2f0426c320e407a5c158170b72"
server_key_version: "encryptionkey@0"
I20260808 18:31:51.564966 5520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 18:31:51.567716 5540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:51.569044 5520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:31:51.569217 5520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "9b2d6ac756f54a959363750fb11e6b47"
format_stamp: "Formatted at 2026-08-08 18:31:51 on dist-test-slave-jztv"
server_key: "c6d4c47cc781e7c83261827c12dc42b7"
server_key_iv: "88618c2f0426c320e407a5c158170b72"
server_key_version: "encryptionkey@0"
I20260808 18:31:51.569391 5520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:51.574671 5520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:51.578362 5520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:51.578589 5520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:51.586505 5520 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41025
I20260808 18:31:51.586565 5613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41025 every 8 connection(s)
I20260808 18:31:51.587632 5520 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:51.590667 5614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:51.590788 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 5520
I20260808 18:31:51.590962 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:51.591233 7825 external_mini_cluster.cc:1428] Setting key ecfeee56edabcde2184ba85638f6689d
I20260808 18:31:51.596680 5618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:51.592582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58549 (local address 127.0.0.1:41025)
0808 18:31:51.593005 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:51.593020 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:51.593057 (+ 37us) server_negotiation.cc:407] Connection header received
0808 18:31:51.593880 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:51.593900 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:51.594187 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:51.594558 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:51.595459 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:51.595463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:51.595475 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:51.595487 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:51.595539 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:51.595558 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:51.595670 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:51.595795 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 18:31:51.595967 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":133,"threads_started":1}
I20260808 18:31:51.597817 5614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47: Bootstrap starting.
I20260808 18:31:51.600220 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:31:51.600603 5614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:51.601375 5614 log.cc:824] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:51.603773 5614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47: No bootstrap required, opened a new log
I20260808 18:31:51.604372 5618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:51.602527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57925 (local address 127.0.0.1:41025)
0808 18:31:51.602653 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:51.602658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:51.602842 (+ 184us) server_negotiation.cc:407] Connection header received
0808 18:31:51.603026 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:51.603034 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:51.603067 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:51.603150 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:51.603968 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:51.603971 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:51.603973 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:51.603984 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:51.604008 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:51.604019 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:51.604114 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:51.604208 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 18:31:51.604248 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:31:51.607954 5614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2d6ac756f54a959363750fb11e6b47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } }
I20260808 18:31:51.608186 5614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:51.608249 5614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b2d6ac756f54a959363750fb11e6b47, State: Initialized, Role: FOLLOWER
I20260808 18:31:51.608698 5614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2d6ac756f54a959363750fb11e6b47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } }
I20260808 18:31:51.608848 5614 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:51.608943 5614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:51.609063 5614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:51.610005 5614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2d6ac756f54a959363750fb11e6b47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } }
I20260808 18:31:51.610407 5614 leader_election.cc:302] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b2d6ac756f54a959363750fb11e6b47; no voters:
I20260808 18:31:51.610759 5614 leader_election.cc:288] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:51.610904 5627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:51.611174 5627 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [term 1 LEADER]: Becoming Leader. State: Replica: 9b2d6ac756f54a959363750fb11e6b47, State: Running, Role: LEADER
I20260808 18:31:51.611510 5627 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2d6ac756f54a959363750fb11e6b47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } }
I20260808 18:31:51.612020 5614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:51.613300 5630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b2d6ac756f54a959363750fb11e6b47. Latest consensus state: current_term: 1 leader_uuid: "9b2d6ac756f54a959363750fb11e6b47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2d6ac756f54a959363750fb11e6b47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } } }
I20260808 18:31:51.613325 5629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b2d6ac756f54a959363750fb11e6b47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2d6ac756f54a959363750fb11e6b47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } } }
I20260808 18:31:51.613447 5630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:51.613449 5629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:51.614501 5641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:51.617076 5641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:51.622546 5641 catalog_manager.cc:1347] Generated new cluster ID: 3c4ad39231ce4736b0c610e99275582e
I20260808 18:31:51.622611 5641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:51.629730 5641 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:51.630026 5641 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:51.636301 5641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b2d6ac756f54a959363750fb11e6b47: Generated new TSK 0
I20260808 18:31:51.637020 5641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:31:51.649052 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 5520
2026-08-08T18:31:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1614 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:51 dist-test-slave-jztv krb5kdc[5657](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:51 dist-test-slave-jztv krb5kdc[5657](info): set up 2 sockets
Aug 08 18:31:51 dist-test-slave-jztv krb5kdc[5657](info): commencing operation
krb5kdc: starting...
W20260808 18:31:52.891423 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:31:52 dist-test-slave-jztv krb5kdc[5657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:31:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:31:53.022181 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40565
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:53.141510 5674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:53.141791 5674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:53.141834 5674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:53.146283 5674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:53.146353 5674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:53.146374 5674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:53.146392 5674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:53.146430 5674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:53.152055 5674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:53.153160 5674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:53.154333 5674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:53.163498 5684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:53.163523 5687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:53.163524 5685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:53.163957 5674 server_base.cc:1034] running on GCE node
I20260808 18:31:53.164502 5674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:53.165624 5674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:53.166831 5674 hybrid_clock.cc:648] HybridClock initialized: now 1786213913166800 us; error 43 us; skew 500 ppm
Aug 08 18:31:53 dist-test-slave-jztv krb5kdc[5657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213913, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:53.170014 5674 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:31:53.171053 5674 webserver.cc:460] Webserver started at http://127.0.0.1:32999/ using document root <none> and password file <none>
I20260808 18:31:53.171636 5674 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:53.171729 5674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:53.171950 5674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:53.174679 5674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "d259ed61391445c29606b58c2542c8fa"
format_stamp: "Formatted at 2026-08-08 18:31:53 on dist-test-slave-jztv"
server_key: "66024678abc390ee581f61b0351c52e7"
server_key_iv: "adcef2e37748698738fbb7818e08bf59"
server_key_version: "encryptionkey@0"
I20260808 18:31:53.175271 5674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "d259ed61391445c29606b58c2542c8fa"
format_stamp: "Formatted at 2026-08-08 18:31:53 on dist-test-slave-jztv"
server_key: "66024678abc390ee581f61b0351c52e7"
server_key_iv: "adcef2e37748698738fbb7818e08bf59"
server_key_version: "encryptionkey@0"
I20260808 18:31:53.179148 5674 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:31:53.181789 5695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:53.183115 5674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:31:53.183264 5674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "d259ed61391445c29606b58c2542c8fa"
format_stamp: "Formatted at 2026-08-08 18:31:53 on dist-test-slave-jztv"
server_key: "66024678abc390ee581f61b0351c52e7"
server_key_iv: "adcef2e37748698738fbb7818e08bf59"
server_key_version: "encryptionkey@0"
I20260808 18:31:53.183390 5674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:53.200021 5674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:53.203392 5674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:53.203610 5674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:53.211246 5674 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40565
I20260808 18:31:53.211261 5768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40565 every 8 connection(s)
I20260808 18:31:53.212231 5674 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:53.215212 5770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:53.220013 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 5674
I20260808 18:31:53.220170 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:53.220455 7825 external_mini_cluster.cc:1428] Setting key 4c286c5281e9bac472354b9a1f3678cd
I20260808 18:31:53.221197 5770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa: Bootstrap starting.
I20260808 18:31:53.223637 5770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:53.224732 5770 log.cc:824] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:53.227061 5770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa: No bootstrap required, opened a new log
Aug 08 18:31:53 dist-test-slave-jztv krb5kdc[5657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 18:31:53.231827 5770 raft_consensus.cc:348] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d259ed61391445c29606b58c2542c8fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40565 } }
I20260808 18:31:53.232151 5770 raft_consensus.cc:374] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:53.232213 5770 raft_consensus.cc:729] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d259ed61391445c29606b58c2542c8fa, State: Initialized, Role: FOLLOWER
I20260808 18:31:53.232725 5770 consensus_queue.cc:260] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d259ed61391445c29606b58c2542c8fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40565 } }
I20260808 18:31:53.232900 5770 raft_consensus.cc:388] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:53.232975 5770 raft_consensus.cc:482] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:53.233081 5770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:53.234254 5770 raft_consensus.cc:504] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d259ed61391445c29606b58c2542c8fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40565 } }
I20260808 18:31:53.234623 5770 leader_election.cc:302] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d259ed61391445c29606b58c2542c8fa; no voters:
I20260808 18:31:53.234970 5770 leader_election.cc:288] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:53.235091 5778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:53.235457 5778 raft_consensus.cc:686] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [term 1 LEADER]: Becoming Leader. State: Replica: d259ed61391445c29606b58c2542c8fa, State: Running, Role: LEADER
I20260808 18:31:53.235853 5778 consensus_queue.cc:237] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d259ed61391445c29606b58c2542c8fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40565 } }
I20260808 18:31:53.236322 5770 sys_catalog.cc:564] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:53.236560 5774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:53.221873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:34535 (local address 127.0.0.1:40565)
0808 18:31:53.222432 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:53.222457 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:53.222493 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:31:53.223139 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:53.223156 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:53.223520 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:53.223879 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:53.224946 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:53.226167 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:53.227407 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:53.227874 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:53.228001 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:53.230954 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:53.230964 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:53.230982 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:53.231016 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:53.233065 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:53.233615 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:53.233621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:53.233627 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:53.233766 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:53.234171 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:53.234177 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:53.234182 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:53.234727 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:53.234960 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:53.235356 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 18:31:53.235584 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":139,"threads_started":1}
I20260808 18:31:53.237614 5782 sys_catalog.cc:455] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d259ed61391445c29606b58c2542c8fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d259ed61391445c29606b58c2542c8fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40565 } } }
I20260808 18:31:53.238045 5782 sys_catalog.cc:458] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:53.237936 5780 sys_catalog.cc:455] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d259ed61391445c29606b58c2542c8fa. Latest consensus state: current_term: 1 leader_uuid: "d259ed61391445c29606b58c2542c8fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d259ed61391445c29606b58c2542c8fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40565 } } }
I20260808 18:31:53.238601 5780 sys_catalog.cc:458] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:53.239946 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 18:31:53.241552 5804 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:31:53.241619 5804 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:31:53.241714 5803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:53.242702 5803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:53.249475 5803 catalog_manager.cc:1347] Generated new cluster ID: 9984c576ecd24e8b92a6474a70f07690
I20260808 18:31:53.249536 5803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:53.250905 5774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:53.242164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:35443 (local address 127.0.0.1:40565)
0808 18:31:53.242307 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:53.242311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:53.242572 (+ 261us) server_negotiation.cc:407] Connection header received
0808 18:31:53.242713 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:53.242716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:53.242793 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:53.242873 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:53.243596 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:53.244112 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:53.245220 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:53.245553 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:53.245647 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:53.247457 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:53.247460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:53.247462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:53.247483 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:53.248781 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:53.249337 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:53.249340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:53.249342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:53.249443 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:53.249842 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:53.249846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:53.249847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:53.250077 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:53.250273 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:53.250635 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 18:31:53.250741 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 18:31:53.261562 5803 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:53.262554 5803 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:53.267395 5803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d259ed61391445c29606b58c2542c8fa: Generated new TSK 0
I20260808 18:31:53.268072 5803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:31:53.276129 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 5674
W20260808 18:31:53.287637 5655 connection.cc:535] client connection to 127.0.0.1:40565 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40565): Connection reset by peer (error 104)
2026-08-08T18:31:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1623 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:53 dist-test-slave-jztv krb5kdc[5821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:53 dist-test-slave-jztv krb5kdc[5821](info): set up 2 sockets
Aug 08 18:31:53 dist-test-slave-jztv krb5kdc[5821](info): commencing operation
krb5kdc: starting...
W20260808 18:31:54.505369 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:31:54 dist-test-slave-jztv krb5kdc[5821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:31:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:31:54.639722 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:54.756016 5835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:54.756400 5835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:54.756518 5835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:54.760650 5835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:54.760761 5835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:54.760802 5835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:54.760840 5835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:54.760877 5835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:54.766314 5835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:54.767490 5835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:54.768548 5835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:54.777324 5843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:54.777315 5844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:54.777508 5835 server_base.cc:1034] running on GCE node
W20260808 18:31:54.777318 5846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:54.778205 5835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:54.779455 5835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:54.780634 5835 hybrid_clock.cc:648] HybridClock initialized: now 1786213914780615 us; error 32 us; skew 500 ppm
Aug 08 18:31:54 dist-test-slave-jztv krb5kdc[5821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213914, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:54.783592 5835 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:31:54.784548 5835 webserver.cc:460] Webserver started at http://127.0.0.1:43397/ using document root <none> and password file <none>
I20260808 18:31:54.785115 5835 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:54.785174 5835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:54.785392 5835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:54.787504 5835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "24ead65d0f6742bf9c29537c809db18c"
format_stamp: "Formatted at 2026-08-08 18:31:54 on dist-test-slave-jztv"
server_key: "fff63ab0016b56369e5f2380154a0927"
server_key_iv: "ff941c17302e48ccece932be339c5787"
server_key_version: "encryptionkey@0"
I20260808 18:31:54.788103 5835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "24ead65d0f6742bf9c29537c809db18c"
format_stamp: "Formatted at 2026-08-08 18:31:54 on dist-test-slave-jztv"
server_key: "fff63ab0016b56369e5f2380154a0927"
server_key_iv: "ff941c17302e48ccece932be339c5787"
server_key_version: "encryptionkey@0"
I20260808 18:31:54.791818 5835 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:31:54.794354 5859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:54.795565 5835 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:31:54.795698 5835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "24ead65d0f6742bf9c29537c809db18c"
format_stamp: "Formatted at 2026-08-08 18:31:54 on dist-test-slave-jztv"
server_key: "fff63ab0016b56369e5f2380154a0927"
server_key_iv: "ff941c17302e48ccece932be339c5787"
server_key_version: "encryptionkey@0"
I20260808 18:31:54.795810 5835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:54.808784 5835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:54.812156 5835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:54.812372 5835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:54.820184 5835 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33607
I20260808 18:31:54.820214 5933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33607 every 8 connection(s)
I20260808 18:31:54.821202 5835 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:54.824220 5936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:54.827508 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 5835
I20260808 18:31:54.827674 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:54.827984 7825 external_mini_cluster.cc:1428] Setting key d5dc109a2b417c1cb47509aa3f60230d
I20260808 18:31:54.830331 5936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c: Bootstrap starting.
I20260808 18:31:54.832741 5936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:54.833712 5936 log.cc:824] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:54.836000 5936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c: No bootstrap required, opened a new log
I20260808 18:31:54.837625 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:54.829567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:39425 (local address 127.0.0.1:33607)
0808 18:31:54.830016 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:54.830033 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:54.830070 (+ 37us) server_negotiation.cc:407] Connection header received
0808 18:31:54.830841 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:54.830859 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:54.831209 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:54.831492 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:54.831889 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:54.832933 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:54.834188 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:54.834830 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:54.834959 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:54.835863 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:54.835868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:54.835881 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:54.836336 (+ 455us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:54.836390 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:54.836414 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:54.836523 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:54.836748 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 18:31:54.836896 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":153,"threads_started":1}
I20260808 18:31:54.839874 5936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ead65d0f6742bf9c29537c809db18c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33607 } }
I20260808 18:31:54.840179 5936 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:54.840265 5936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24ead65d0f6742bf9c29537c809db18c, State: Initialized, Role: FOLLOWER
I20260808 18:31:54.840677 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:31:54.840802 5936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ead65d0f6742bf9c29537c809db18c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33607 } }
I20260808 18:31:54.840971 5936 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:54.841023 5936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:54.841152 5936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:54.842145 5936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ead65d0f6742bf9c29537c809db18c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33607 } }
I20260808 18:31:54.842527 5945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:31:54.842538 5936 leader_election.cc:302] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24ead65d0f6742bf9c29537c809db18c; no voters:
I20260808 18:31:54.842646 5945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ead65d0f6742bf9c29537c809db18c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33607 } }
I20260808 18:31:54.842945 5936 leader_election.cc:288] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:54.843076 5951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:54.843173 5945 leader_election.cc:302] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24ead65d0f6742bf9c29537c809db18c; no voters:
I20260808 18:31:54.843307 5951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [term 1 LEADER]: Becoming Leader. State: Replica: 24ead65d0f6742bf9c29537c809db18c, State: Running, Role: LEADER
I20260808 18:31:54.843358 5945 leader_election.cc:288] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers
I20260808 18:31:54.843618 5951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ead65d0f6742bf9c29537c809db18c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33607 } }
I20260808 18:31:54.844244 5936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:54.844940 5945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24ead65d0f6742bf9c29537c809db18c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ead65d0f6742bf9c29537c809db18c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33607 } } }
I20260808 18:31:54.845072 5945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:54.845592 5960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24ead65d0f6742bf9c29537c809db18c. Latest consensus state: current_term: 1 leader_uuid: "24ead65d0f6742bf9c29537c809db18c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ead65d0f6742bf9c29537c809db18c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33607 } } }
I20260808 18:31:54.845760 5960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c [sys.catalog]: This master's current role is: LEADER
W20260808 18:31:54.849061 5980 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:31:54.849192 5980 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:31:54.849301 5969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:54.850240 5969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:54.850674 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:54.843188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:47257 (local address 127.0.0.1:33607)
0808 18:31:54.843305 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:54.843309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:54.844931 (+ 1622us) server_negotiation.cc:407] Connection header received
0808 18:31:54.845129 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:54.845133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:54.845187 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:54.845245 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:54.845724 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:54.846515 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:54.847572 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:54.847905 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:54.848011 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:54.849098 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:54.849101 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:54.849105 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:54.849117 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:54.849144 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:54.849158 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:54.849237 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:54.850469 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0808 18:31:54.850505 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 18:31:54.855960 5969 catalog_manager.cc:1347] Generated new cluster ID: eda1b44ecdd34d249abbd15b9240cbd3
I20260808 18:31:54.856038 5969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:54.880673 5969 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:54.881906 5969 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:54.890620 5969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24ead65d0f6742bf9c29537c809db18c: Generated new TSK 0
I20260808 18:31:54.891330 5969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:31:54.897578 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 5835
W20260808 18:31:54.908692 5817 connection.cc:535] client connection to 127.0.0.1:33607 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33607): Connection reset by peer (error 104)
2026-08-08T18:31:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1621 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:54 dist-test-slave-jztv krb5kdc[5992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:54 dist-test-slave-jztv krb5kdc[5992](info): set up 2 sockets
Aug 08 18:31:54 dist-test-slave-jztv krb5kdc[5992](info): commencing operation
krb5kdc: starting...
W20260808 18:31:56.111241 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:31:56 dist-test-slave-jztv krb5kdc[5992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:31:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:31:56.245114 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42923
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:56.360486 6007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:56.360812 6007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:56.360903 6007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:56.364979 6007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:56.365067 6007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:56.365104 6007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:56.365161 6007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:56.365198 6007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:56.370499 6007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:56.371551 6007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:56.372565 6007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:56.381307 6016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:56.381294 6017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:56.381500 6007 server_base.cc:1034] running on GCE node
W20260808 18:31:56.381309 6019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:56.382282 6007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:56.383383 6007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:56.384586 6007 hybrid_clock.cc:648] HybridClock initialized: now 1786213916384567 us; error 30 us; skew 500 ppm
Aug 08 18:31:56 dist-test-slave-jztv krb5kdc[5992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213916, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:56.390952 6007 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:31:56.391924 6007 webserver.cc:460] Webserver started at http://127.0.0.1:35899/ using document root <none> and password file <none>
I20260808 18:31:56.392520 6007 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:56.392616 6007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:56.392858 6007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:56.395002 6007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "cd1842ff28e84bbe91c29d93b5c0be68"
format_stamp: "Formatted at 2026-08-08 18:31:56 on dist-test-slave-jztv"
server_key: "17dd30314718e2d76c6fc580cab23109"
server_key_iv: "37e77096626306a682fc898aadcb29fd"
server_key_version: "encryptionkey@0"
I20260808 18:31:56.395584 6007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "cd1842ff28e84bbe91c29d93b5c0be68"
format_stamp: "Formatted at 2026-08-08 18:31:56 on dist-test-slave-jztv"
server_key: "17dd30314718e2d76c6fc580cab23109"
server_key_iv: "37e77096626306a682fc898aadcb29fd"
server_key_version: "encryptionkey@0"
I20260808 18:31:56.399228 6007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:31:56.401590 6029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:56.402817 6007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:31:56.402966 6007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "cd1842ff28e84bbe91c29d93b5c0be68"
format_stamp: "Formatted at 2026-08-08 18:31:56 on dist-test-slave-jztv"
server_key: "17dd30314718e2d76c6fc580cab23109"
server_key_iv: "37e77096626306a682fc898aadcb29fd"
server_key_version: "encryptionkey@0"
I20260808 18:31:56.403079 6007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:56.407634 6007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:56.410415 6007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:56.410609 6007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:56.417693 6007 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42923
I20260808 18:31:56.417723 6105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42923 every 8 connection(s)
I20260808 18:31:56.418665 6007 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:56.421314 6106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:56.421990 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 6007
I20260808 18:31:56.422134 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:56.422447 7825 external_mini_cluster.cc:1428] Setting key 3df71a1b6d32c8fd4645efaae0981b23
I20260808 18:31:56.427543 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:56.423815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:41195 (local address 127.0.0.1:42923)
0808 18:31:56.424236 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:56.424251 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:56.424285 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:31:56.424940 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:56.424957 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:56.425227 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:56.425558 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:56.426416 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:56.426421 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:56.426438 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:56.426450 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:56.426505 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:56.426524 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:56.426676 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:56.426746 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 18:31:56.426903 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":129,"threads_started":1}
I20260808 18:31:56.427611 6106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68: Bootstrap starting.
I20260808 18:31:56.429885 6106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:56.430266 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:31:56.430773 6106 log.cc:824] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:56.432932 6106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68: No bootstrap required, opened a new log
I20260808 18:31:56.434156 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:56.432390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:39807 (local address 127.0.0.1:42923)
0808 18:31:56.432512 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:56.432516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:56.432685 (+ 169us) server_negotiation.cc:407] Connection header received
0808 18:31:56.432803 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:56.432805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:56.432832 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:56.432894 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:56.433692 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:56.433695 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:56.433697 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:56.433708 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:56.433742 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:56.433753 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:56.433832 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:56.433966 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 18:31:56.434006 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:31:56.436707 6106 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd1842ff28e84bbe91c29d93b5c0be68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42923 } }
I20260808 18:31:56.436928 6106 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:56.437008 6106 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1842ff28e84bbe91c29d93b5c0be68, State: Initialized, Role: FOLLOWER
I20260808 18:31:56.437428 6106 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd1842ff28e84bbe91c29d93b5c0be68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42923 } }
I20260808 18:31:56.437577 6106 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:56.437644 6106 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:56.437765 6106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:56.438722 6106 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd1842ff28e84bbe91c29d93b5c0be68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42923 } }
I20260808 18:31:56.439054 6106 leader_election.cc:302] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd1842ff28e84bbe91c29d93b5c0be68; no voters:
I20260808 18:31:56.439374 6106 leader_election.cc:288] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:56.439479 6117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:56.439715 6117 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [term 1 LEADER]: Becoming Leader. State: Replica: cd1842ff28e84bbe91c29d93b5c0be68, State: Running, Role: LEADER
I20260808 18:31:56.440097 6117 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd1842ff28e84bbe91c29d93b5c0be68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42923 } }
I20260808 18:31:56.440618 6106 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:56.441877 6119 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd1842ff28e84bbe91c29d93b5c0be68. Latest consensus state: current_term: 1 leader_uuid: "cd1842ff28e84bbe91c29d93b5c0be68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd1842ff28e84bbe91c29d93b5c0be68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42923 } } }
I20260808 18:31:56.441886 6118 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd1842ff28e84bbe91c29d93b5c0be68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd1842ff28e84bbe91c29d93b5c0be68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42923 } } }
I20260808 18:31:56.442039 6119 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:56.442041 6118 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:56.442471 6128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:56.445076 6128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:56.449999 6128 catalog_manager.cc:1347] Generated new cluster ID: c74fcfff7f9a4683a1e7e7e4ca2b7868
I20260808 18:31:56.450067 6128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:56.468190 6128 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:56.468524 6128 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:56.474941 6128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd1842ff28e84bbe91c29d93b5c0be68: Generated new TSK 0
I20260808 18:31:56.475716 6128 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:31:56.485343 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 6007
2026-08-08T18:31:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1588 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:56 dist-test-slave-jztv krb5kdc[6143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:56 dist-test-slave-jztv krb5kdc[6143](info): set up 2 sockets
Aug 08 18:31:56 dist-test-slave-jztv krb5kdc[6143](info): commencing operation
krb5kdc: starting...
W20260808 18:31:57.707059 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:31:57 dist-test-slave-jztv krb5kdc[6143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:31:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:31:57.836714 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36383
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:57.952136 6160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:57.952414 6160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:57.952457 6160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:57.956414 6160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:57.956477 6160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:57.956496 6160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:57.956513 6160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:57.956528 6160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:57.961968 6160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36383
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:57.963063 6160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:57.964052 6160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:57.972664 6167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:57.972664 6166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:57.972688 6172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:57.972990 6160 server_base.cc:1034] running on GCE node
I20260808 18:31:57.973610 6160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:57.974836 6160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:57.976056 6160 hybrid_clock.cc:648] HybridClock initialized: now 1786213917976020 us; error 48 us; skew 500 ppm
Aug 08 18:31:57 dist-test-slave-jztv krb5kdc[6143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213917, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:57.978948 6160 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:31:57.979825 6160 webserver.cc:460] Webserver started at http://127.0.0.1:45005/ using document root <none> and password file <none>
I20260808 18:31:57.980340 6160 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:57.980398 6160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:57.980579 6160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:57.982697 6160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "6cbb2aba770541cc808424db6a3e38e6"
format_stamp: "Formatted at 2026-08-08 18:31:57 on dist-test-slave-jztv"
server_key: "6409c7fce7111981b3b35338da324eda"
server_key_iv: "9f11d4a13d72d78085b4329cadf1a6e7"
server_key_version: "encryptionkey@0"
I20260808 18:31:57.983218 6160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "6cbb2aba770541cc808424db6a3e38e6"
format_stamp: "Formatted at 2026-08-08 18:31:57 on dist-test-slave-jztv"
server_key: "6409c7fce7111981b3b35338da324eda"
server_key_iv: "9f11d4a13d72d78085b4329cadf1a6e7"
server_key_version: "encryptionkey@0"
I20260808 18:31:57.987004 6160 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:31:57.989506 6179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:57.990792 6160 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:31:57.990924 6160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "6cbb2aba770541cc808424db6a3e38e6"
format_stamp: "Formatted at 2026-08-08 18:31:57 on dist-test-slave-jztv"
server_key: "6409c7fce7111981b3b35338da324eda"
server_key_iv: "9f11d4a13d72d78085b4329cadf1a6e7"
server_key_version: "encryptionkey@0"
I20260808 18:31:57.991036 6160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:58.003697 6160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:58.006625 6160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:58.006829 6160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:58.014326 6160 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36383
I20260808 18:31:58.014370 6254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36383 every 8 connection(s)
I20260808 18:31:58.015362 6160 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:58.018728 6255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:58.024233 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 6160
I20260808 18:31:58.024397 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:58.024717 7825 external_mini_cluster.cc:1428] Setting key 4e23edd6cd3b33ab99997912f01864f0
I20260808 18:31:58.025743 6255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6: Bootstrap starting.
I20260808 18:31:58.028304 6255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:58.029321 6255 log.cc:824] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:58.031666 6255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6: No bootstrap required, opened a new log
Aug 08 18:31:58 dist-test-slave-jztv krb5kdc[6143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 18:31:58.036300 6255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cbb2aba770541cc808424db6a3e38e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260808 18:31:58.036671 6255 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:58.036763 6255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cbb2aba770541cc808424db6a3e38e6, State: Initialized, Role: FOLLOWER
I20260808 18:31:58.037261 6255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cbb2aba770541cc808424db6a3e38e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260808 18:31:58.037418 6255 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:58.037501 6255 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:58.037606 6255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:58.038952 6255 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cbb2aba770541cc808424db6a3e38e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260808 18:31:58.039451 6255 leader_election.cc:302] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cbb2aba770541cc808424db6a3e38e6; no voters:
I20260808 18:31:58.039963 6255 leader_election.cc:288] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:58.040084 6264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:58.040380 6264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [term 1 LEADER]: Becoming Leader. State: Replica: 6cbb2aba770541cc808424db6a3e38e6, State: Running, Role: LEADER
I20260808 18:31:58.040771 6264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cbb2aba770541cc808424db6a3e38e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260808 18:31:58.041419 6255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:58.041543 6262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:58.026365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:52401 (local address 127.0.0.1:36383)
0808 18:31:58.026839 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:58.026863 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:58.026907 (+ 44us) server_negotiation.cc:407] Connection header received
0808 18:31:58.027598 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:58.027616 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:58.027989 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:58.028276 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:58.029287 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:58.030458 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:58.031636 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:58.032210 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:58.032346 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:58.035410 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:58.035416 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:58.035431 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:58.035470 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:58.037458 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:58.038115 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:58.038120 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:58.038125 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:58.038326 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:58.038711 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:58.038714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:58.038715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:58.039234 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:58.039419 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:58.040126 (+ 707us) server_negotiation.cc:299] Negotiation successful
0808 18:31:58.040442 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":182,"threads_started":1}
I20260808 18:31:58.042632 6267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cbb2aba770541cc808424db6a3e38e6. Latest consensus state: current_term: 1 leader_uuid: "6cbb2aba770541cc808424db6a3e38e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cbb2aba770541cc808424db6a3e38e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } } }
I20260808 18:31:58.042764 6267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:58.043200 6274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:58.043159 6265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cbb2aba770541cc808424db6a3e38e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cbb2aba770541cc808424db6a3e38e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } } }
I20260808 18:31:58.043246 6265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:58.045418 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:31:58.046550 6274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:58.054381 6274 catalog_manager.cc:1347] Generated new cluster ID: e1def64454bc436c9c8e18bbe1c46b5c
I20260808 18:31:58.054453 6274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:58.057873 6262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:58.047913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:59725 (local address 127.0.0.1:36383)
0808 18:31:58.048061 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:58.048066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:58.049557 (+ 1491us) server_negotiation.cc:407] Connection header received
0808 18:31:58.049630 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:58.049632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:58.049677 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:58.049800 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:58.050592 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:58.051129 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:58.051972 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:58.052207 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:58.052316 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:58.054133 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:58.054137 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:58.054140 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:31:58.054171 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:58.055941 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:58.056504 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:58.056508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:58.056510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:58.056601 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:31:58.056943 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:31:58.056947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:31:58.056949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:31:58.057175 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:58.057328 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:58.057616 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 18:31:58.057715 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:31:58.061957 6274 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:58.063119 6274 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:58.070309 6274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cbb2aba770541cc808424db6a3e38e6: Generated new TSK 0
I20260808 18:31:58.071097 6274 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:31:58.078204 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 6160
W20260808 18:31:58.094722 6140 connection.cc:535] client connection to 127.0.0.1:36383 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36383): Connection reset by peer (error 104)
2026-08-08T18:31:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1597 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:58 dist-test-slave-jztv krb5kdc[6301](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:58 dist-test-slave-jztv krb5kdc[6301](info): set up 2 sockets
Aug 08 18:31:58 dist-test-slave-jztv krb5kdc[6301](info): commencing operation
krb5kdc: starting...
W20260808 18:31:59.306879 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:31:59 dist-test-slave-jztv krb5kdc[6301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:31:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:31:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:31:59.439178 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35179
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:31:59.552749 6319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:31:59.553017 6319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:31:59.553058 6319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:31:59.557090 6319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:31:59.557147 6319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:31:59.557166 6319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:31:59.557183 6319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:31:59.557199 6319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:31:59.562533 6319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35179
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:31:59.563524 6319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:31:59.564493 6319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:31:59.573103 6331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:59.573139 6328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:31:59.573155 6325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:31:59.573295 6319 server_base.cc:1034] running on GCE node
I20260808 18:31:59.573940 6319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:31:59.575088 6319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:31:59.576282 6319 hybrid_clock.cc:648] HybridClock initialized: now 1786213919576275 us; error 29 us; skew 500 ppm
Aug 08 18:31:59 dist-test-slave-jztv krb5kdc[6301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:31:59.579181 6319 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:31:59.580107 6319 webserver.cc:460] Webserver started at http://127.0.0.1:40273/ using document root <none> and password file <none>
I20260808 18:31:59.580677 6319 fs_manager.cc:359] Metadata directory not provided
I20260808 18:31:59.580772 6319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:31:59.581017 6319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:31:59.583144 6319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "0bf67ff6ccdc427eb7e09a46a6eba638"
format_stamp: "Formatted at 2026-08-08 18:31:59 on dist-test-slave-jztv"
server_key: "1df08bc49e18a7752a7c8e9b42e0506a"
server_key_iv: "476c7b7b3753e43fadab1344b6f0fb12"
server_key_version: "encryptionkey@0"
I20260808 18:31:59.583737 6319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "0bf67ff6ccdc427eb7e09a46a6eba638"
format_stamp: "Formatted at 2026-08-08 18:31:59 on dist-test-slave-jztv"
server_key: "1df08bc49e18a7752a7c8e9b42e0506a"
server_key_iv: "476c7b7b3753e43fadab1344b6f0fb12"
server_key_version: "encryptionkey@0"
I20260808 18:31:59.587741 6319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:31:59.590453 6341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:31:59.591749 6319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:31:59.591895 6319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "0bf67ff6ccdc427eb7e09a46a6eba638"
format_stamp: "Formatted at 2026-08-08 18:31:59 on dist-test-slave-jztv"
server_key: "1df08bc49e18a7752a7c8e9b42e0506a"
server_key_iv: "476c7b7b3753e43fadab1344b6f0fb12"
server_key_version: "encryptionkey@0"
I20260808 18:31:59.592024 6319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:31:59.619187 6319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:31:59.622771 6319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:31:59.622994 6319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:31:59.631035 6319 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35179
I20260808 18:31:59.631065 6424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35179 every 8 connection(s)
I20260808 18:31:59.632055 6319 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:31:59.635084 6426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:31:59.637112 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 6319
I20260808 18:31:59.637255 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:31:59.637573 7825 external_mini_cluster.cc:1428] Setting key 37daa1eeb4328d5f0056a4b168ca7a40
I20260808 18:31:59.641885 6426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638: Bootstrap starting.
I20260808 18:31:59.644239 6426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638: Neither blocks nor log segments found. Creating new log.
I20260808 18:31:59.644997 6426 log.cc:824] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638: Log is configured to *not* fsync() on all Append() calls
I20260808 18:31:59.646826 6429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:59.639156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:44593 (local address 127.0.0.1:35179)
0808 18:31:59.639635 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:59.639651 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:59.639687 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:31:59.640331 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:59.640349 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:59.640706 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:59.641051 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:59.641461 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:59.642562 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:59.643447 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:59.643925 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:59.644118 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:59.645038 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:59.645044 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:59.645057 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:59.645489 (+ 432us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:59.645547 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:59.645571 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:59.645718 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:59.645911 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 18:31:59.646065 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":157,"threads_started":1}
I20260808 18:31:59.647140 6426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638: No bootstrap required, opened a new log
I20260808 18:31:59.649787 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:31:59.651082 6426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf67ff6ccdc427eb7e09a46a6eba638" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35179 } }
I20260808 18:31:59.651341 6426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:31:59.651432 6426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf67ff6ccdc427eb7e09a46a6eba638, State: Initialized, Role: FOLLOWER
I20260808 18:31:59.651943 6426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf67ff6ccdc427eb7e09a46a6eba638" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35179 } }
I20260808 18:31:59.652143 6426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:31:59.652237 6426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:31:59.652338 6426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:31:59.653246 6426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf67ff6ccdc427eb7e09a46a6eba638" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35179 } }
I20260808 18:31:59.653632 6426 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bf67ff6ccdc427eb7e09a46a6eba638; no voters:
I20260808 18:31:59.653965 6426 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:31:59.654088 6439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:31:59.654378 6439 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [term 1 LEADER]: Becoming Leader. State: Replica: 0bf67ff6ccdc427eb7e09a46a6eba638, State: Running, Role: LEADER
I20260808 18:31:59.654736 6439 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf67ff6ccdc427eb7e09a46a6eba638" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35179 } }
I20260808 18:31:59.655368 6426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:31:59.656239 6439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bf67ff6ccdc427eb7e09a46a6eba638. Latest consensus state: current_term: 1 leader_uuid: "0bf67ff6ccdc427eb7e09a46a6eba638" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf67ff6ccdc427eb7e09a46a6eba638" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35179 } } }
I20260808 18:31:59.656803 6429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:31:59.651908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:48963 (local address 127.0.0.1:35179)
0808 18:31:59.652034 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 18:31:59.652043 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:31:59.652269 (+ 226us) server_negotiation.cc:407] Connection header received
0808 18:31:59.652406 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:31:59.652409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:31:59.652464 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:31:59.652534 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:31:59.652802 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:59.653330 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:59.654482 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:31:59.654895 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:31:59.655001 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:31:59.656052 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:31:59.656056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:31:59.656059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:31:59.656072 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:31:59.656099 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:31:59.656112 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:31:59.656199 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 18:31:59.656581 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 18:31:59.656624 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:31:59.657567 6439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [sys.catalog]: This master's current role is: LEADER
I20260808 18:31:59.656911 6446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bf67ff6ccdc427eb7e09a46a6eba638" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf67ff6ccdc427eb7e09a46a6eba638" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35179 } } }
I20260808 18:31:59.658075 6446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638 [sys.catalog]: This master's current role is: LEADER
W20260808 18:31:59.660696 6462 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:31:59.660763 6462 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:31:59.660847 6463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:31:59.661664 6463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:31:59.667153 6463 catalog_manager.cc:1347] Generated new cluster ID: 3b822cbc8c7542fb9ce10e2f7824efa4
I20260808 18:31:59.667240 6463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:31:59.684382 6463 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:31:59.685379 6463 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:31:59.691541 6463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bf67ff6ccdc427eb7e09a46a6eba638: Generated new TSK 0
I20260808 18:31:59.692420 6463 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:31:59.706509 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 6319
W20260808 18:31:59.716113 6298 connection.cc:535] client connection to 127.0.0.1:35179 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35179): Connection reset by peer (error 104)
2026-08-08T18:31:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1621 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:31:59 dist-test-slave-jztv krb5kdc[6472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:31:59 dist-test-slave-jztv krb5kdc[6472](info): set up 2 sockets
Aug 08 18:31:59 dist-test-slave-jztv krb5kdc[6472](info): commencing operation
krb5kdc: starting...
W20260808 18:32:00.994882 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:32:01 dist-test-slave-jztv krb5kdc[6472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:32:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:32:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:32:01.125499 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40981
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:32:01.239069 6488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:32:01.239346 6488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:32:01.239389 6488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:32:01.243513 6488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:32:01.243573 6488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:32:01.243592 6488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:32:01.243608 6488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:32:01.243623 6488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:32:01.248806 6488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40981
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:32:01.249817 6488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:32:01.250851 6488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:32:01.259548 6494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:01.259574 6497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:01.259615 6488 server_base.cc:1034] running on GCE node
W20260808 18:32:01.259600 6493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:01.260462 6488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:32:01.261540 6488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:32:01.262703 6488 hybrid_clock.cc:648] HybridClock initialized: now 1786213921262688 us; error 26 us; skew 500 ppm
Aug 08 18:32:01 dist-test-slave-jztv krb5kdc[6472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213921, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:32:01.265489 6488 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:32:01.266461 6488 webserver.cc:460] Webserver started at http://127.0.0.1:41081/ using document root <none> and password file <none>
I20260808 18:32:01.267143 6488 fs_manager.cc:359] Metadata directory not provided
I20260808 18:32:01.267210 6488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:32:01.267459 6488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:32:01.269547 6488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "c149e7b4dc2b4b8e90805da656b17d77"
format_stamp: "Formatted at 2026-08-08 18:32:01 on dist-test-slave-jztv"
server_key: "2af0a114d9250051844bc214f745e2dd"
server_key_iv: "c1ad9172e4e6329750cb5e91118db234"
server_key_version: "encryptionkey@0"
I20260808 18:32:01.270151 6488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "c149e7b4dc2b4b8e90805da656b17d77"
format_stamp: "Formatted at 2026-08-08 18:32:01 on dist-test-slave-jztv"
server_key: "2af0a114d9250051844bc214f745e2dd"
server_key_iv: "c1ad9172e4e6329750cb5e91118db234"
server_key_version: "encryptionkey@0"
I20260808 18:32:01.273996 6488 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:32:01.276485 6510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:32:01.277727 6488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:32:01.277876 6488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "c149e7b4dc2b4b8e90805da656b17d77"
format_stamp: "Formatted at 2026-08-08 18:32:01 on dist-test-slave-jztv"
server_key: "2af0a114d9250051844bc214f745e2dd"
server_key_iv: "c1ad9172e4e6329750cb5e91118db234"
server_key_version: "encryptionkey@0"
I20260808 18:32:01.277995 6488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:32:01.282821 6488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:32:01.285663 6488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:32:01.285866 6488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:32:01.293313 6488 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40981
I20260808 18:32:01.293344 6598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40981 every 8 connection(s)
I20260808 18:32:01.294377 6488 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:32:01.297122 6599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:32:01.302320 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 6488
I20260808 18:32:01.302467 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:32:01.302778 7825 external_mini_cluster.cc:1428] Setting key 00da8b3ef30f2a7bae61e83edd6fc8f7
I20260808 18:32:01.302891 6599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77: Bootstrap starting.
I20260808 18:32:01.305415 6599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77: Neither blocks nor log segments found. Creating new log.
I20260808 18:32:01.306170 6599 log.cc:824] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77: Log is configured to *not* fsync() on all Append() calls
I20260808 18:32:01.308075 6602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:01.304202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:58141 (local address 127.0.0.1:40981)
0808 18:32:01.304665 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:01.304681 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:01.304716 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:32:01.305351 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:01.305367 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:01.305726 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:01.306076 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:01.306903 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:01.306908 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:01.306918 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:32:01.306927 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:01.306971 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:32:01.306988 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:01.307120 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:01.307226 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 18:32:01.307396 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":140,"threads_started":1}
I20260808 18:32:01.308270 6599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77: No bootstrap required, opened a new log
I20260808 18:32:01.311009 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:32:01.312517 6599 raft_consensus.cc:348] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c149e7b4dc2b4b8e90805da656b17d77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40981 } }
I20260808 18:32:01.312764 6599 raft_consensus.cc:374] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:32:01.312834 6599 raft_consensus.cc:729] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c149e7b4dc2b4b8e90805da656b17d77, State: Initialized, Role: FOLLOWER
I20260808 18:32:01.313319 6599 consensus_queue.cc:260] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c149e7b4dc2b4b8e90805da656b17d77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40981 } }
I20260808 18:32:01.313520 6599 raft_consensus.cc:388] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:32:01.313607 6599 raft_consensus.cc:482] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:32:01.313741 6599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:32:01.315117 6602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:01.313137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:53189 (local address 127.0.0.1:40981)
0808 18:32:01.313272 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:01.313276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:01.313502 (+ 226us) server_negotiation.cc:407] Connection header received
0808 18:32:01.313647 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:01.313650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:01.313709 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:01.313788 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:01.314677 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:01.314680 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:01.314682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:32:01.314693 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:01.314719 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:32:01.314730 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:01.314806 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:01.314931 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 18:32:01.314968 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:32:01.315116 6599 raft_consensus.cc:504] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c149e7b4dc2b4b8e90805da656b17d77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40981 } }
I20260808 18:32:01.315570 6599 leader_election.cc:302] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c149e7b4dc2b4b8e90805da656b17d77; no voters:
I20260808 18:32:01.315954 6599 leader_election.cc:288] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:32:01.316286 6609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:32:01.316476 6609 raft_consensus.cc:686] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [term 1 LEADER]: Becoming Leader. State: Replica: c149e7b4dc2b4b8e90805da656b17d77, State: Running, Role: LEADER
I20260808 18:32:01.316780 6609 consensus_queue.cc:237] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c149e7b4dc2b4b8e90805da656b17d77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40981 } }
I20260808 18:32:01.317339 6599 sys_catalog.cc:564] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:32:01.318603 6610 sys_catalog.cc:455] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c149e7b4dc2b4b8e90805da656b17d77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c149e7b4dc2b4b8e90805da656b17d77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40981 } } }
I20260808 18:32:01.318745 6610 sys_catalog.cc:458] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:01.319065 6612 sys_catalog.cc:455] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c149e7b4dc2b4b8e90805da656b17d77. Latest consensus state: current_term: 1 leader_uuid: "c149e7b4dc2b4b8e90805da656b17d77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c149e7b4dc2b4b8e90805da656b17d77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40981 } } }
I20260808 18:32:01.319164 6612 sys_catalog.cc:458] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77 [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:01.319275 6620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:32:01.322566 6620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:32:01.328316 6620 catalog_manager.cc:1347] Generated new cluster ID: ff4c7b67786e4679ab8be657067cb9a9
I20260808 18:32:01.328410 6620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:32:01.350659 6620 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:32:01.351035 6620 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:32:01.357159 6620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c149e7b4dc2b4b8e90805da656b17d77: Generated new TSK 0
I20260808 18:32:01.357980 6620 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:32:01.371667 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 6488
2026-08-08T18:32:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1671 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:32:01 dist-test-slave-jztv krb5kdc[6637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:32:01 dist-test-slave-jztv krb5kdc[6637](info): set up 2 sockets
Aug 08 18:32:01 dist-test-slave-jztv krb5kdc[6637](info): commencing operation
krb5kdc: starting...
W20260808 18:32:02.629796 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:32:02 dist-test-slave-jztv krb5kdc[6637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:32:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:32:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:32:02.766362 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:40121
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:36375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:40121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:32:02.882675 6661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:32:02.882999 6661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:32:02.883086 6661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:32:02.887517 6661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:32:02.887624 6661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:32:02.887661 6661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:32:02.887698 6661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:32:02.887733 6661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:32:02.893671 6661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:36375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:40121
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:40121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:32:02.894773 6661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:32:02.895817 6661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:32:02.904513 6669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:02.904553 6672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:02.904513 6670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:02.905020 6661 server_base.cc:1034] running on GCE node
I20260808 18:32:02.905599 6661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:32:02.906843 6661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:32:02.908047 6661 hybrid_clock.cc:648] HybridClock initialized: now 1786213922908037 us; error 40 us; skew 500 ppm
Aug 08 18:32:02 dist-test-slave-jztv krb5kdc[6637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213922, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:32:02.910883 6661 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:32:02.911793 6661 webserver.cc:460] Webserver started at http://127.7.164.126:35861/ using document root <none> and password file <none>
I20260808 18:32:02.912366 6661 fs_manager.cc:359] Metadata directory not provided
I20260808 18:32:02.912492 6661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:32:02.912724 6661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:32:02.914901 6661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "44490e0213314180a9e2d044879bd5f9"
format_stamp: "Formatted at 2026-08-08 18:32:02 on dist-test-slave-jztv"
server_key: "44892e8349209d783bee6c88c9bd808c"
server_key_iv: "aefac69c355ce814d4f05988be755e0b"
server_key_version: "encryptionkey@0"
I20260808 18:32:02.915503 6661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "44490e0213314180a9e2d044879bd5f9"
format_stamp: "Formatted at 2026-08-08 18:32:02 on dist-test-slave-jztv"
server_key: "44892e8349209d783bee6c88c9bd808c"
server_key_iv: "aefac69c355ce814d4f05988be755e0b"
server_key_version: "encryptionkey@0"
I20260808 18:32:02.919317 6661 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 18:32:02.921749 6687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:32:02.923080 6661 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:32:02.923228 6661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "44490e0213314180a9e2d044879bd5f9"
format_stamp: "Formatted at 2026-08-08 18:32:02 on dist-test-slave-jztv"
server_key: "44892e8349209d783bee6c88c9bd808c"
server_key_iv: "aefac69c355ce814d4f05988be755e0b"
server_key_version: "encryptionkey@0"
I20260808 18:32:02.923350 6661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:32:02.940907 6661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:32:02.944099 6661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:32:02.944319 6661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:32:02.951762 6661 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:40121
I20260808 18:32:02.951792 6762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:40121 every 8 connection(s)
I20260808 18:32:02.952734 6661 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:32:02.954504 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 6661
I20260808 18:32:02.954654 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:32:02.954962 7825 external_mini_cluster.cc:1428] Setting key 6ea304a9630ab75211c446a2e397aaa6
I20260808 18:32:02.955852 6763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:32:02.961694 6763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9: Bootstrap starting.
Aug 08 18:32:02 dist-test-slave-jztv krb5kdc[6637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:32:02.964380 6763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9: Neither blocks nor log segments found. Creating new log.
I20260808 18:32:02.965269 6763 log.cc:824] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9: Log is configured to *not* fsync() on all Append() calls
I20260808 18:32:02.967352 6763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9: No bootstrap required, opened a new log
I20260808 18:32:02.969381 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:02.956379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43267 (local address 127.7.164.126:40121)
0808 18:32:02.956848 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:02.956864 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:02.956900 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:32:02.957552 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:02.957567 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:02.957892 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:02.958265 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:02.959121 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:02.960149 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:02.961062 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:02.961539 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:02.961716 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:02.964491 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:02.964499 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:02.964521 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:02.964550 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:02.966523 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:02.967049 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:02.967060 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:02.967064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:02.967173 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:02.967525 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:02.967535 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:02.967537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:02.967918 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:02.968148 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:02.968460 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 18:32:02.968688 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":167,"threads_started":1}
I20260808 18:32:02.971544 6763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44490e0213314180a9e2d044879bd5f9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40121 } }
I20260808 18:32:02.971822 6763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:32:02.971887 6763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44490e0213314180a9e2d044879bd5f9, State: Initialized, Role: FOLLOWER
I20260808 18:32:02.972353 6763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44490e0213314180a9e2d044879bd5f9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40121 } }
I20260808 18:32:02.972584 6763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:32:02.972635 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:32:02.972663 6763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:32:02.972755 6763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:32:02.973718 6763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44490e0213314180a9e2d044879bd5f9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40121 } }
I20260808 18:32:02.974097 6763 leader_election.cc:302] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44490e0213314180a9e2d044879bd5f9; no voters:
I20260808 18:32:02.974505 6763 leader_election.cc:288] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:32:02.974586 6779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:32:02.974804 6779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [term 1 LEADER]: Becoming Leader. State: Replica: 44490e0213314180a9e2d044879bd5f9, State: Running, Role: LEADER
I20260808 18:32:02.975121 6779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44490e0213314180a9e2d044879bd5f9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40121 } }
I20260808 18:32:02.975858 6763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:32:02.976336 6779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44490e0213314180a9e2d044879bd5f9. Latest consensus state: current_term: 1 leader_uuid: "44490e0213314180a9e2d044879bd5f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44490e0213314180a9e2d044879bd5f9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40121 } } }
I20260808 18:32:02.976454 6779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:02.976672 6779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44490e0213314180a9e2d044879bd5f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44490e0213314180a9e2d044879bd5f9" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 40121 } } }
I20260808 18:32:02.976779 6779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9 [sys.catalog]: This master's current role is: LEADER
W20260808 18:32:02.981276 6804 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:32:02.981419 6804 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:32:02.981573 6785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:32:02.982679 6785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:32:02.984432 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:02.974777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53729 (local address 127.7.164.126:40121)
0808 18:32:02.974967 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:02.974971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:02.975199 (+ 228us) server_negotiation.cc:407] Connection header received
0808 18:32:02.975386 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:02.975389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:02.975440 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:02.975524 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:02.976292 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:02.977065 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:02.978204 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:02.978551 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:02.978630 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:02.980309 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:02.980313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:02.980315 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:02.980338 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:02.981948 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:02.982608 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:02.982613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:02.982615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:02.982708 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:02.983147 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:02.983150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:02.983152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:02.983369 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:02.983507 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:02.984172 (+ 665us) server_negotiation.cc:299] Negotiation successful
0808 18:32:02.984272 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:32:02.988780 6785 catalog_manager.cc:1347] Generated new cluster ID: ddb09e6b9bba4c32943e372ae9e63f12
I20260808 18:32:02.988869 6785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:32:03.009806 6785 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:32:03.010928 6785 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:32:03.016306 6785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44490e0213314180a9e2d044879bd5f9: Generated new TSK 0
I20260808 18:32:03.017024 6785 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:32:03.024454 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 6661
W20260808 18:32:03.038033 6635 connection.cc:535] client connection to 127.7.164.126:40121 recv error: Network error: failed to read from TLS socket (remote: 127.7.164.126:40121): Connection reset by peer (error 104)
2026-08-08T18:32:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1650 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:32:03 dist-test-slave-jztv krb5kdc[6814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:32:03 dist-test-slave-jztv krb5kdc[6814](info): set up 2 sockets
Aug 08 18:32:03 dist-test-slave-jztv krb5kdc[6814](info): commencing operation
krb5kdc: starting...
W20260808 18:32:04.251176 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:32:04 dist-test-slave-jztv krb5kdc[6814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:32:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:32:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:32:04.385711 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:32949
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:32949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:32:04.499756 6831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:32:04.500106 6831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:32:04.500195 6831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:32:04.504340 6831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:32:04.504462 6831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:32:04.504503 6831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:32:04.504541 6831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:32:04.504575 6831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:32:04.510067 6831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:32949
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:32949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:32:04.511237 6831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:32:04.512310 6831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:32:04.521270 6836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:04.521394 6831 server_base.cc:1034] running on GCE node
W20260808 18:32:04.521263 6842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:04.521263 6837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:04.522158 6831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:32:04.523372 6831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:32:04.524588 6831 hybrid_clock.cc:648] HybridClock initialized: now 1786213924524567 us; error 33 us; skew 500 ppm
Aug 08 18:32:04 dist-test-slave-jztv krb5kdc[6814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213924, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:32:04.527617 6831 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:32:04.528575 6831 webserver.cc:460] Webserver started at http://127.7.164.126:41213/ using document root <none> and password file <none>
I20260808 18:32:04.529107 6831 fs_manager.cc:359] Metadata directory not provided
I20260808 18:32:04.529168 6831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:32:04.529341 6831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:32:04.531457 6831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "9d3b6484bfe84d2a9758d72f8dec3a99"
format_stamp: "Formatted at 2026-08-08 18:32:04 on dist-test-slave-jztv"
server_key: "612a34a929a7bbca730d6f76bee6a6aa"
server_key_iv: "38a2b6c2bcdca5056f177a560f42075d"
server_key_version: "encryptionkey@0"
I20260808 18:32:04.531989 6831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "9d3b6484bfe84d2a9758d72f8dec3a99"
format_stamp: "Formatted at 2026-08-08 18:32:04 on dist-test-slave-jztv"
server_key: "612a34a929a7bbca730d6f76bee6a6aa"
server_key_iv: "38a2b6c2bcdca5056f177a560f42075d"
server_key_version: "encryptionkey@0"
I20260808 18:32:04.535900 6831 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:32:04.538453 6852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:32:04.539660 6831 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:32:04.539808 6831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "9d3b6484bfe84d2a9758d72f8dec3a99"
format_stamp: "Formatted at 2026-08-08 18:32:04 on dist-test-slave-jztv"
server_key: "612a34a929a7bbca730d6f76bee6a6aa"
server_key_iv: "38a2b6c2bcdca5056f177a560f42075d"
server_key_version: "encryptionkey@0"
I20260808 18:32:04.539927 6831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:32:04.557878 6831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:32:04.561352 6831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:32:04.561573 6831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:32:04.569121 6831 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:32949
I20260808 18:32:04.569147 6924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:32949 every 8 connection(s)
I20260808 18:32:04.570097 6831 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:32:04.572896 6925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:32:04.572969 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 6831
I20260808 18:32:04.573136 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:32:04.573453 7825 external_mini_cluster.cc:1428] Setting key 4b001e83038d91e05927455c94cc8c80
I20260808 18:32:04.579599 6925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99: Bootstrap starting.
I20260808 18:32:04.581979 6925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99: Neither blocks nor log segments found. Creating new log.
I20260808 18:32:04.582707 6928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:04.574885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38397 (local address 127.7.164.126:32949)
0808 18:32:04.575361 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:04.575376 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:04.575414 (+ 38us) server_negotiation.cc:407] Connection header received
0808 18:32:04.576265 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:04.576283 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:04.576612 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:04.576955 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:04.577349 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:04.578399 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:04.579309 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:04.579840 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:04.580018 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:04.580964 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:04.580970 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:04.580982 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:32:04.581360 (+ 378us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:04.581416 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:32:04.581438 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:04.581582 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:04.581748 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 18:32:04.581892 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":134,"threads_started":1}
I20260808 18:32:04.582922 6925 log.cc:824] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99: Log is configured to *not* fsync() on all Append() calls
I20260808 18:32:04.585167 6925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99: No bootstrap required, opened a new log
I20260808 18:32:04.585704 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:32:04.589115 6925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3b6484bfe84d2a9758d72f8dec3a99" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32949 } }
I20260808 18:32:04.589342 6925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:32:04.589414 6925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3b6484bfe84d2a9758d72f8dec3a99, State: Initialized, Role: FOLLOWER
I20260808 18:32:04.589879 6925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3b6484bfe84d2a9758d72f8dec3a99" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32949 } }
I20260808 18:32:04.590042 6925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:32:04.590132 6925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:32:04.590276 6925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:32:04.591320 6925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3b6484bfe84d2a9758d72f8dec3a99" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32949 } }
I20260808 18:32:04.591681 6925 leader_election.cc:302] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d3b6484bfe84d2a9758d72f8dec3a99; no voters:
I20260808 18:32:04.592043 6925 leader_election.cc:288] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:32:04.592113 6938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:32:04.592270 6928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:04.587913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33969 (local address 127.7.164.126:32949)
0808 18:32:04.588031 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:04.588038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:04.588218 (+ 180us) server_negotiation.cc:407] Connection header received
0808 18:32:04.588391 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:04.588394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:04.588435 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:04.588518 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:04.588811 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:04.589376 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:04.590392 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:04.590639 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:04.590750 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:04.591709 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:04.591713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:04.591716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:32:04.591727 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:04.591752 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:32:04.591784 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:04.591878 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:04.592002 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 18:32:04.592038 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:32:04.592334 6938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [term 1 LEADER]: Becoming Leader. State: Replica: 9d3b6484bfe84d2a9758d72f8dec3a99, State: Running, Role: LEADER
I20260808 18:32:04.592633 6938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3b6484bfe84d2a9758d72f8dec3a99" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32949 } }
I20260808 18:32:04.593379 6925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:32:04.593839 6938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d3b6484bfe84d2a9758d72f8dec3a99. Latest consensus state: current_term: 1 leader_uuid: "9d3b6484bfe84d2a9758d72f8dec3a99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3b6484bfe84d2a9758d72f8dec3a99" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32949 } } }
I20260808 18:32:04.593961 6938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:04.594386 6940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d3b6484bfe84d2a9758d72f8dec3a99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3b6484bfe84d2a9758d72f8dec3a99" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32949 } } }
I20260808 18:32:04.594496 6940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99 [sys.catalog]: This master's current role is: LEADER
W20260808 18:32:04.598764 6961 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:32:04.598853 6961 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:32:04.598959 6944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:32:04.599700 6944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:32:04.605101 6944 catalog_manager.cc:1347] Generated new cluster ID: b96318afdef3425eb990492de56ae44f
I20260808 18:32:04.605173 6944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:32:04.628865 6944 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:32:04.629845 6944 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:32:04.634253 6944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d3b6484bfe84d2a9758d72f8dec3a99: Generated new TSK 0
I20260808 18:32:04.634929 6944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:32:04.640726 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 6831
W20260808 18:32:04.652221 6810 connection.cc:535] client connection to 127.7.164.126:32949 recv error: Network error: failed to read from TLS socket (remote: 127.7.164.126:32949): Connection reset by peer (error 104)
2026-08-08T18:32:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1613 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:32:04 dist-test-slave-jztv krb5kdc[6967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:32:04 dist-test-slave-jztv krb5kdc[6967](info): set up 2 sockets
Aug 08 18:32:04 dist-test-slave-jztv krb5kdc[6967](info): commencing operation
krb5kdc: starting...
W20260808 18:32:05.870926 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:32:05 dist-test-slave-jztv krb5kdc[6967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:32:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:32:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:32:06.003696 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:42787
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:42787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:32:06.118471 6983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:32:06.118781 6983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:32:06.118855 6983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:32:06.122828 6983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:32:06.122915 6983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:32:06.122953 6983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:32:06.123008 6983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:32:06.123049 6983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:32:06.128098 6983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:42787
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:42787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:32:06.129091 6983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:32:06.130069 6983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:32:06.138482 6992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:06.138491 6990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:06.138494 6994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:06.138742 6983 server_base.cc:1034] running on GCE node
I20260808 18:32:06.139314 6983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:32:06.140393 6983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:32:06.141625 6983 hybrid_clock.cc:648] HybridClock initialized: now 1786213926141600 us; error 37 us; skew 500 ppm
Aug 08 18:32:06 dist-test-slave-jztv krb5kdc[6967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213926, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:32:06.144510 6983 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:32:06.145488 6983 webserver.cc:460] Webserver started at http://127.7.164.126:42243/ using document root <none> and password file <none>
I20260808 18:32:06.146075 6983 fs_manager.cc:359] Metadata directory not provided
I20260808 18:32:06.146175 6983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:32:06.146448 6983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:32:06.148496 6983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "7ca796efb17d4fd0a8dde5af6db42fa6"
format_stamp: "Formatted at 2026-08-08 18:32:06 on dist-test-slave-jztv"
server_key: "123c75f2dfe2e86049e6e159e7532e73"
server_key_iv: "0bbb1c5b94c75b1fc25a6dfdad5d4033"
server_key_version: "encryptionkey@0"
I20260808 18:32:06.149098 6983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "7ca796efb17d4fd0a8dde5af6db42fa6"
format_stamp: "Formatted at 2026-08-08 18:32:06 on dist-test-slave-jztv"
server_key: "123c75f2dfe2e86049e6e159e7532e73"
server_key_iv: "0bbb1c5b94c75b1fc25a6dfdad5d4033"
server_key_version: "encryptionkey@0"
I20260808 18:32:06.152894 6983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 18:32:06.155342 7007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:32:06.156489 6983 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 18:32:06.156642 6983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "7ca796efb17d4fd0a8dde5af6db42fa6"
format_stamp: "Formatted at 2026-08-08 18:32:06 on dist-test-slave-jztv"
server_key: "123c75f2dfe2e86049e6e159e7532e73"
server_key_iv: "0bbb1c5b94c75b1fc25a6dfdad5d4033"
server_key_version: "encryptionkey@0"
I20260808 18:32:06.156759 6983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:32:06.161589 6983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:32:06.164826 6983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:32:06.165025 6983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:32:06.172916 6983 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:42787
I20260808 18:32:06.172946 7079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:42787 every 8 connection(s)
I20260808 18:32:06.173844 6983 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:32:06.176515 7080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:32:06.180940 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 6983
I20260808 18:32:06.181092 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:32:06.181414 7825 external_mini_cluster.cc:1428] Setting key 38165fd8f5c8c24a63cccb73cd790459
I20260808 18:32:06.182449 7080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6: Bootstrap starting.
I20260808 18:32:06.184723 7080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6: Neither blocks nor log segments found. Creating new log.
I20260808 18:32:06.185494 7080 log.cc:824] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6: Log is configured to *not* fsync() on all Append() calls
I20260808 18:32:06.186906 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:06.182854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35611 (local address 127.7.164.126:42787)
0808 18:32:06.183314 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:06.183330 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:06.183364 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:32:06.184003 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:06.184020 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:06.184289 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:06.184637 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:06.185654 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:06.185659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:06.185673 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:32:06.185684 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:06.185739 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:32:06.185759 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:06.185889 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:06.186008 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 18:32:06.186170 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":149,"threads_started":1}
I20260808 18:32:06.187462 7080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6: No bootstrap required, opened a new log
I20260808 18:32:06.189657 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:32:06.191300 7080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca796efb17d4fd0a8dde5af6db42fa6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 42787 } }
I20260808 18:32:06.191545 7080 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:32:06.191612 7080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca796efb17d4fd0a8dde5af6db42fa6, State: Initialized, Role: FOLLOWER
I20260808 18:32:06.192087 7080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca796efb17d4fd0a8dde5af6db42fa6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 42787 } }
I20260808 18:32:06.192270 7080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:32:06.192350 7080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:32:06.192457 7080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:32:06.193490 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:06.191693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40955 (local address 127.7.164.126:42787)
0808 18:32:06.191811 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:06.191816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:06.192048 (+ 232us) server_negotiation.cc:407] Connection header received
0808 18:32:06.192126 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:06.192129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:06.192164 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:06.192252 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:06.193038 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:06.193042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:06.193044 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:32:06.193056 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:06.193085 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:32:06.193098 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:06.193188 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:06.193335 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 18:32:06.193378 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:32:06.193694 7080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca796efb17d4fd0a8dde5af6db42fa6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 42787 } }
I20260808 18:32:06.194025 7080 leader_election.cc:302] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ca796efb17d4fd0a8dde5af6db42fa6; no voters:
I20260808 18:32:06.194397 7080 leader_election.cc:288] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:32:06.194442 7090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:32:06.194725 7090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [term 1 LEADER]: Becoming Leader. State: Replica: 7ca796efb17d4fd0a8dde5af6db42fa6, State: Running, Role: LEADER
I20260808 18:32:06.195065 7090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca796efb17d4fd0a8dde5af6db42fa6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 42787 } }
I20260808 18:32:06.195627 7080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:32:06.196766 7091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ca796efb17d4fd0a8dde5af6db42fa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca796efb17d4fd0a8dde5af6db42fa6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 42787 } } }
I20260808 18:32:06.196908 7091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:06.197206 7092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ca796efb17d4fd0a8dde5af6db42fa6. Latest consensus state: current_term: 1 leader_uuid: "7ca796efb17d4fd0a8dde5af6db42fa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca796efb17d4fd0a8dde5af6db42fa6" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 42787 } } }
I20260808 18:32:06.197297 7100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:32:06.198239 7092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:06.200296 7100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:32:06.205533 7100 catalog_manager.cc:1347] Generated new cluster ID: 2a59c6b63a0645f6affe9434dc187e42
I20260808 18:32:06.205593 7100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:32:06.219250 7100 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:32:06.219563 7100 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:32:06.223626 7100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ca796efb17d4fd0a8dde5af6db42fa6: Generated new TSK 0
I20260808 18:32:06.224225 7100 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:32:06.229279 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 6983
2026-08-08T18:32:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1591 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:32:06 dist-test-slave-jztv krb5kdc[7115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:32:06 dist-test-slave-jztv krb5kdc[7115](info): set up 2 sockets
Aug 08 18:32:06 dist-test-slave-jztv krb5kdc[7115](info): commencing operation
krb5kdc: starting...
W20260808 18:32:07.451684 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:32:07 dist-test-slave-jztv krb5kdc[7115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:32:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:32:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:32:07.589547 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:35967
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:33579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:35967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:32:07.702584 7133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:32:07.702914 7133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:32:07.702992 7133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:32:07.707024 7133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:32:07.707127 7133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:32:07.707167 7133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:32:07.707227 7133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:32:07.707267 7133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:32:07.712580 7133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:33579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:35967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:35967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:32:07.713673 7133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:32:07.714840 7133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:32:07.723686 7144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:07.723663 7142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:07.723671 7140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:07.723908 7133 server_base.cc:1034] running on GCE node
I20260808 18:32:07.724551 7133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:32:07.725641 7133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:32:07.726801 7133 hybrid_clock.cc:648] HybridClock initialized: now 1786213927726786 us; error 26 us; skew 500 ppm
Aug 08 18:32:07 dist-test-slave-jztv krb5kdc[7115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213927, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:32:07.733299 7133 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:32:07.734331 7133 webserver.cc:460] Webserver started at http://127.7.164.126:43625/ using document root <none> and password file <none>
I20260808 18:32:07.734897 7133 fs_manager.cc:359] Metadata directory not provided
I20260808 18:32:07.734959 7133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:32:07.735210 7133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:32:07.737239 7133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "ee1cd8f3a59043adbd9cc38705b3727b"
format_stamp: "Formatted at 2026-08-08 18:32:07 on dist-test-slave-jztv"
server_key: "cffc657f979bb4da82106741a798f896"
server_key_iv: "c4443facef8d58efbc41900b8536cab1"
server_key_version: "encryptionkey@0"
I20260808 18:32:07.737884 7133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "ee1cd8f3a59043adbd9cc38705b3727b"
format_stamp: "Formatted at 2026-08-08 18:32:07 on dist-test-slave-jztv"
server_key: "cffc657f979bb4da82106741a798f896"
server_key_iv: "c4443facef8d58efbc41900b8536cab1"
server_key_version: "encryptionkey@0"
I20260808 18:32:07.741617 7133 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:32:07.744065 7151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:32:07.745292 7133 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:32:07.745438 7133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "ee1cd8f3a59043adbd9cc38705b3727b"
format_stamp: "Formatted at 2026-08-08 18:32:07 on dist-test-slave-jztv"
server_key: "cffc657f979bb4da82106741a798f896"
server_key_iv: "c4443facef8d58efbc41900b8536cab1"
server_key_version: "encryptionkey@0"
I20260808 18:32:07.745553 7133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:32:07.769076 7133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:32:07.772473 7133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:32:07.772710 7133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:32:07.780241 7133 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:35967
I20260808 18:32:07.780268 7225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:35967 every 8 connection(s)
I20260808 18:32:07.781250 7133 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:32:07.784186 7226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:32:07.787176 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 7133
I20260808 18:32:07.787328 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:32:07.787635 7825 external_mini_cluster.cc:1428] Setting key e5d64f55bdb19ef0a83a4d6b8db2d2bc
I20260808 18:32:07.790184 7226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b: Bootstrap starting.
I20260808 18:32:07.792363 7226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b: Neither blocks nor log segments found. Creating new log.
I20260808 18:32:07.793094 7226 log.cc:824] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b: Log is configured to *not* fsync() on all Append() calls
I20260808 18:32:07.794991 7226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b: No bootstrap required, opened a new log
Aug 08 18:32:07 dist-test-slave-jztv krb5kdc[7115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:32:07.799661 7226 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee1cd8f3a59043adbd9cc38705b3727b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35967 } }
I20260808 18:32:07.799934 7226 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:32:07.799999 7226 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1cd8f3a59043adbd9cc38705b3727b, State: Initialized, Role: FOLLOWER
I20260808 18:32:07.800496 7226 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee1cd8f3a59043adbd9cc38705b3727b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35967 } }
I20260808 18:32:07.800661 7226 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:32:07.800729 7226 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:32:07.800822 7226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:32:07.801988 7226 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee1cd8f3a59043adbd9cc38705b3727b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35967 } }
I20260808 18:32:07.802374 7226 leader_election.cc:302] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee1cd8f3a59043adbd9cc38705b3727b; no voters:
I20260808 18:32:07.802718 7226 leader_election.cc:288] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:32:07.802770 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:07.789117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59851 (local address 127.7.164.126:35967)
0808 18:32:07.789566 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:07.789582 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:07.789617 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:32:07.790261 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:07.790278 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:07.790664 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:07.790988 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:07.791828 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:07.792882 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:07.793792 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:07.794338 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:07.794513 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:07.797340 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:07.797347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:07.797363 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:07.797394 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:07.799435 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:07.799867 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:07.799873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:07.799878 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:07.800039 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:07.800383 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:07.800386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:07.800388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:07.800942 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:07.801141 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:07.801411 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 18:32:07.801726 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":139,"threads_started":1}
I20260808 18:32:07.802861 7232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:32:07.803058 7232 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [term 1 LEADER]: Becoming Leader. State: Replica: ee1cd8f3a59043adbd9cc38705b3727b, State: Running, Role: LEADER
I20260808 18:32:07.803375 7232 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee1cd8f3a59043adbd9cc38705b3727b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35967 } }
I20260808 18:32:07.803995 7226 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:32:07.805112 7235 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee1cd8f3a59043adbd9cc38705b3727b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee1cd8f3a59043adbd9cc38705b3727b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35967 } } }
I20260808 18:32:07.805244 7235 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:07.805621 7234 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee1cd8f3a59043adbd9cc38705b3727b. Latest consensus state: current_term: 1 leader_uuid: "ee1cd8f3a59043adbd9cc38705b3727b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee1cd8f3a59043adbd9cc38705b3727b" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 35967 } } }
I20260808 18:32:07.805711 7234 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:07.806046 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:32:07.806569 7244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:32:07.809438 7244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:32:07.815287 7244 catalog_manager.cc:1347] Generated new cluster ID: c437bc6859ea4f63b511d7148ad2592c
I20260808 18:32:07.815371 7244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:32:07.819932 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:07.808503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36023 (local address 127.7.164.126:35967)
0808 18:32:07.808641 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:07.808645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:07.810589 (+ 1944us) server_negotiation.cc:407] Connection header received
0808 18:32:07.810719 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:07.810723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:07.810775 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:07.810863 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:07.811753 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:07.812572 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:07.813673 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:07.814585 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:07.814708 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:07.816453 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:07.816458 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:07.816461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:07.816488 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:07.818144 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:07.818642 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:07.818645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:07.818648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:07.818744 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:07.819030 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:07.819035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:07.819039 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:07.819264 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:07.819424 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:07.819645 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 18:32:07.819772 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:32:07.853022 7244 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:32:07.853979 7244 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:32:07.866101 7244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee1cd8f3a59043adbd9cc38705b3727b: Generated new TSK 0
I20260808 18:32:07.866876 7244 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:32:07.873344 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 7133
2026-08-08T18:32:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1644 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:32:07 dist-test-slave-jztv krb5kdc[7271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:32:07 dist-test-slave-jztv krb5kdc[7271](info): set up 2 sockets
Aug 08 18:32:07 dist-test-slave-jztv krb5kdc[7271](info): commencing operation
krb5kdc: starting...
W20260808 18:32:09.095520 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:32:09 dist-test-slave-jztv krb5kdc[7271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:32:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:32:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:32:09.229435 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:41881
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:41881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:32:09.341516 7289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:32:09.341821 7289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:32:09.341897 7289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:32:09.345894 7289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:32:09.345983 7289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:32:09.346016 7289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:32:09.346058 7289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:32:09.346088 7289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:32:09.351330 7289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:41881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:41881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:32:09.352361 7289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:32:09.353359 7289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:32:09.361805 7302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:09.361860 7298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:09.361814 7300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:09.362279 7289 server_base.cc:1034] running on GCE node
I20260808 18:32:09.362851 7289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:32:09.363920 7289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:32:09.365082 7289 hybrid_clock.cc:648] HybridClock initialized: now 1786213929365063 us; error 31 us; skew 500 ppm
Aug 08 18:32:09 dist-test-slave-jztv krb5kdc[7271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213929, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:32:09.367739 7289 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:32:09.368624 7289 webserver.cc:460] Webserver started at http://127.7.164.126:37781/ using document root <none> and password file <none>
I20260808 18:32:09.369172 7289 fs_manager.cc:359] Metadata directory not provided
I20260808 18:32:09.369268 7289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:32:09.369503 7289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:32:09.371526 7289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "7f2f84da938b49d1952a72934f020b52"
format_stamp: "Formatted at 2026-08-08 18:32:09 on dist-test-slave-jztv"
server_key: "51f78824c00746ed14f41c1034cb7756"
server_key_iv: "de16ba3b6899ad6026a7cc1749edc311"
server_key_version: "encryptionkey@0"
I20260808 18:32:09.372102 7289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "7f2f84da938b49d1952a72934f020b52"
format_stamp: "Formatted at 2026-08-08 18:32:09 on dist-test-slave-jztv"
server_key: "51f78824c00746ed14f41c1034cb7756"
server_key_iv: "de16ba3b6899ad6026a7cc1749edc311"
server_key_version: "encryptionkey@0"
I20260808 18:32:09.375717 7289 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:32:09.378116 7310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:32:09.379309 7289 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:32:09.379464 7289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "7f2f84da938b49d1952a72934f020b52"
format_stamp: "Formatted at 2026-08-08 18:32:09 on dist-test-slave-jztv"
server_key: "51f78824c00746ed14f41c1034cb7756"
server_key_iv: "de16ba3b6899ad6026a7cc1749edc311"
server_key_version: "encryptionkey@0"
I20260808 18:32:09.379575 7289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:32:09.407284 7289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:32:09.410490 7289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:32:09.410706 7289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:32:09.418116 7289 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:41881
I20260808 18:32:09.418134 7380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:41881 every 8 connection(s)
I20260808 18:32:09.419188 7289 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:32:09.422043 7383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:32:09.426410 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 7289
I20260808 18:32:09.426558 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:32:09.426849 7825 external_mini_cluster.cc:1428] Setting key 7bdda20eea2d6cc73ede363a1ee15d7c
I20260808 18:32:09.428272 7383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52: Bootstrap starting.
I20260808 18:32:09.430662 7383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52: Neither blocks nor log segments found. Creating new log.
I20260808 18:32:09.431399 7383 log.cc:824] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52: Log is configured to *not* fsync() on all Append() calls
I20260808 18:32:09.433205 7383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52: No bootstrap required, opened a new log
I20260808 18:32:09.435817 7389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:09.428202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53505 (local address 127.7.164.126:41881)
0808 18:32:09.428679 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:09.428700 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:09.428736 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:32:09.429360 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:09.429376 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:09.429723 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:09.429995 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:09.430397 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:09.431445 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:09.432391 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:09.432863 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:09.433042 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:09.433834 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:09.433839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:09.433852 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:32:09.434300 (+ 448us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:09.434358 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:32:09.434381 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:09.434491 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:09.434638 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 18:32:09.434849 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":179,"threads_started":1}
I20260808 18:32:09.436637 7383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f84da938b49d1952a72934f020b52" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41881 } }
I20260808 18:32:09.436944 7383 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:32:09.437004 7383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f2f84da938b49d1952a72934f020b52, State: Initialized, Role: FOLLOWER
I20260808 18:32:09.437443 7383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f84da938b49d1952a72934f020b52" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41881 } }
I20260808 18:32:09.437637 7383 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:32:09.437716 7383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:32:09.437865 7383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:32:09.438994 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:32:09.438974 7383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f84da938b49d1952a72934f020b52" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41881 } }
I20260808 18:32:09.439311 7383 leader_election.cc:302] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f2f84da938b49d1952a72934f020b52; no voters:
I20260808 18:32:09.439666 7383 leader_election.cc:288] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:32:09.439805 7391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:32:09.440050 7391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [term 1 LEADER]: Becoming Leader. State: Replica: 7f2f84da938b49d1952a72934f020b52, State: Running, Role: LEADER
I20260808 18:32:09.440379 7391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f84da938b49d1952a72934f020b52" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41881 } }
I20260808 18:32:09.440989 7383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:32:09.441525 7391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f2f84da938b49d1952a72934f020b52. Latest consensus state: current_term: 1 leader_uuid: "7f2f84da938b49d1952a72934f020b52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f84da938b49d1952a72934f020b52" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41881 } } }
I20260808 18:32:09.441653 7391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:09.441965 7393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f2f84da938b49d1952a72934f020b52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f84da938b49d1952a72934f020b52" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 41881 } } }
I20260808 18:32:09.442082 7393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52 [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:09.446005 7389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:09.441255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59047 (local address 127.7.164.126:41881)
0808 18:32:09.441370 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:09.441374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:09.441699 (+ 325us) server_negotiation.cc:407] Connection header received
0808 18:32:09.441921 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:09.441927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:09.441987 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:09.442075 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:09.442491 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:09.443288 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:09.444403 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:09.444617 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:09.444736 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:09.445585 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:09.445589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:09.445592 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:32:09.445611 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:09.445636 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:32:09.445646 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:09.445729 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:09.445818 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 18:32:09.445869 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 18:32:09.446090 7417 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:32:09.446182 7417 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:32:09.446269 7405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:32:09.447129 7405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:32:09.452762 7405 catalog_manager.cc:1347] Generated new cluster ID: 9f169029276348568ce9b8004170c165
I20260808 18:32:09.452829 7405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:32:09.467057 7405 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:32:09.468231 7405 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:32:09.476888 7405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f2f84da938b49d1952a72934f020b52: Generated new TSK 0
I20260808 18:32:09.477792 7405 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:32:09.482264 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 7289
2026-08-08T18:32:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1607 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:32:09 dist-test-slave-jztv krb5kdc[7426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:32:09 dist-test-slave-jztv krb5kdc[7426](info): set up 2 sockets
Aug 08 18:32:09 dist-test-slave-jztv krb5kdc[7426](info): commencing operation
krb5kdc: starting...
W20260808 18:32:10.713625 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:32:10 dist-test-slave-jztv krb5kdc[7426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:32:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:32:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:32:10.845512 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:32927
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:38545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:32927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:32:10.958902 7450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:32:10.959242 7450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:32:10.959350 7450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:32:10.963634 7450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:32:10.963740 7450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:32:10.963779 7450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:32:10.963817 7450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:32:10.963852 7450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:32:10.969238 7450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:38545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:32927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:32927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:32:10.970808 7450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:32:10.971899 7450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:32:10.980790 7457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:10.980777 7460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:10.980947 7450 server_base.cc:1034] running on GCE node
W20260808 18:32:10.980762 7463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:10.981601 7450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:32:10.982757 7450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:32:10.983959 7450 hybrid_clock.cc:648] HybridClock initialized: now 1786213930983941 us; error 36 us; skew 500 ppm
Aug 08 18:32:10 dist-test-slave-jztv krb5kdc[7426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213930, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:32:10.986765 7450 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:32:10.987691 7450 webserver.cc:460] Webserver started at http://127.7.164.126:37269/ using document root <none> and password file <none>
I20260808 18:32:10.988255 7450 fs_manager.cc:359] Metadata directory not provided
I20260808 18:32:10.988351 7450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:32:10.988606 7450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:32:10.990700 7450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "cc2795667fb0468cb336214876a2aded"
format_stamp: "Formatted at 2026-08-08 18:32:10 on dist-test-slave-jztv"
server_key: "397d65a01d3941d5775f620b770fdd2a"
server_key_iv: "9fa6a9fa35a80dc7991983340abeb836"
server_key_version: "encryptionkey@0"
I20260808 18:32:10.991274 7450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "cc2795667fb0468cb336214876a2aded"
format_stamp: "Formatted at 2026-08-08 18:32:10 on dist-test-slave-jztv"
server_key: "397d65a01d3941d5775f620b770fdd2a"
server_key_iv: "9fa6a9fa35a80dc7991983340abeb836"
server_key_version: "encryptionkey@0"
I20260808 18:32:10.995018 7450 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:32:10.997339 7471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:32:10.998636 7450 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 18:32:10.998775 7450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "cc2795667fb0468cb336214876a2aded"
format_stamp: "Formatted at 2026-08-08 18:32:10 on dist-test-slave-jztv"
server_key: "397d65a01d3941d5775f620b770fdd2a"
server_key_iv: "9fa6a9fa35a80dc7991983340abeb836"
server_key_version: "encryptionkey@0"
I20260808 18:32:10.998889 7450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:32:11.003532 7450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:32:11.006464 7450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:32:11.006659 7450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:32:11.013844 7450 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:32927
I20260808 18:32:11.013873 7545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:32927 every 8 connection(s)
I20260808 18:32:11.014881 7450 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:32:11.017820 7547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:32:11.022672 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 7450
I20260808 18:32:11.022835 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:32:11.023129 7825 external_mini_cluster.cc:1428] Setting key 13574f8a37136bff5d7548215d25f700
I20260808 18:32:11.023975 7547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded: Bootstrap starting.
I20260808 18:32:11.026924 7547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded: Neither blocks nor log segments found. Creating new log.
I20260808 18:32:11.027920 7547 log.cc:824] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded: Log is configured to *not* fsync() on all Append() calls
I20260808 18:32:11.029218 7550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:11.024668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55611 (local address 127.7.164.126:32927)
0808 18:32:11.025166 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:11.025190 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:11.025241 (+ 51us) server_negotiation.cc:407] Connection header received
0808 18:32:11.026107 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:11.026131 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:11.026538 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:11.026928 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:11.027778 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:11.027785 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:11.027804 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:32:11.027817 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:11.027885 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:32:11.027912 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:11.028040 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:11.028191 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 18:32:11.028361 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":146,"threads_started":1}
I20260808 18:32:11.030093 7547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded: No bootstrap required, opened a new log
I20260808 18:32:11.032413 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:32:11.034482 7547 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2795667fb0468cb336214876a2aded" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32927 } }
I20260808 18:32:11.034785 7547 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:32:11.034912 7547 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc2795667fb0468cb336214876a2aded, State: Initialized, Role: FOLLOWER
I20260808 18:32:11.035454 7547 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2795667fb0468cb336214876a2aded" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32927 } }
I20260808 18:32:11.035619 7547 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:32:11.035730 7547 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:32:11.035843 7547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:32:11.036703 7550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:11.034746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33451 (local address 127.7.164.126:32927)
0808 18:32:11.034881 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:11.034885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:11.035115 (+ 230us) server_negotiation.cc:407] Connection header received
0808 18:32:11.035273 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:11.035276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:11.035313 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:11.035399 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:11.036250 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:11.036253 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:11.036254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:32:11.036265 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:11.036292 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:32:11.036302 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:11.036390 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:11.036493 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 18:32:11.036525 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:32:11.037120 7547 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2795667fb0468cb336214876a2aded" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32927 } }
I20260808 18:32:11.037505 7547 leader_election.cc:302] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc2795667fb0468cb336214876a2aded; no voters:
I20260808 18:32:11.037850 7547 leader_election.cc:288] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:32:11.037955 7565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:32:11.038178 7565 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [term 1 LEADER]: Becoming Leader. State: Replica: cc2795667fb0468cb336214876a2aded, State: Running, Role: LEADER
I20260808 18:32:11.038578 7565 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2795667fb0468cb336214876a2aded" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32927 } }
I20260808 18:32:11.039146 7547 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:32:11.040249 7566 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc2795667fb0468cb336214876a2aded" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2795667fb0468cb336214876a2aded" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32927 } } }
I20260808 18:32:11.040403 7566 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:11.040711 7567 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc2795667fb0468cb336214876a2aded. Latest consensus state: current_term: 1 leader_uuid: "cc2795667fb0468cb336214876a2aded" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc2795667fb0468cb336214876a2aded" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 32927 } } }
I20260808 18:32:11.040812 7579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:32:11.040874 7567 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:11.043993 7579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:32:11.049659 7579 catalog_manager.cc:1347] Generated new cluster ID: 2798223ce5df4a548377b44fd38e8d90
I20260808 18:32:11.049734 7579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:32:11.066407 7579 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:32:11.066776 7579 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:32:11.080227 7579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc2795667fb0468cb336214876a2aded: Generated new TSK 0
I20260808 18:32:11.080981 7579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:32:11.089711 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 7450
2026-08-08T18:32:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1607 ms)
[----------] 18 tests from AuthTokenIssuingTest (29189 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:32:11 dist-test-slave-jztv krb5kdc[7596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:32:11 dist-test-slave-jztv krb5kdc[7596](info): set up 2 sockets
Aug 08 18:32:11 dist-test-slave-jztv krb5kdc[7596](info): commencing operation
krb5kdc: starting...
W20260808 18:32:12.333230 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:32:12 dist-test-slave-jztv krb5kdc[7596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:32:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:32:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:32:12.459416 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:37341
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:37341,127.7.164.125:37493,127.7.164.124:40391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:32:12.571255 7612 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:32:12.571599 7612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:32:12.571674 7612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:32:12.571758 7612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:32:12.575718 7612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:32:12.575804 7612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:32:12.575841 7612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:32:12.575896 7612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:32:12.575932 7612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:32:12.581710 7612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:37341,127.7.164.125:37493,127.7.164.124:40391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:37341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:32:12.582870 7612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:32:12.583902 7612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:32:12.592684 7622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:12.592725 7624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:12.592697 7628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:12.592844 7612 server_base.cc:1034] running on GCE node
I20260808 18:32:12.593513 7612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:32:12.594704 7612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:32:12.595912 7612 hybrid_clock.cc:648] HybridClock initialized: now 1786213932595890 us; error 44 us; skew 500 ppm
Aug 08 18:32:12 dist-test-slave-jztv krb5kdc[7596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213932, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:32:12.598899 7612 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:32:12.599824 7612 webserver.cc:460] Webserver started at http://127.7.164.126:45853/ using document root <none> and password file <none>
I20260808 18:32:12.600411 7612 fs_manager.cc:359] Metadata directory not provided
I20260808 18:32:12.600509 7612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:32:12.600754 7612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:32:12.602927 7612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "9e5c4bd8fda34e9b9607495295339c5d"
format_stamp: "Formatted at 2026-08-08 18:32:12 on dist-test-slave-jztv"
server_key: "a214e27dcb4eae29251cd963a8518df3"
server_key_iv: "5b8b99585c0727d3d00cf802d4afdd9f"
server_key_version: "encryptionkey@0"
I20260808 18:32:12.603596 7612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "9e5c4bd8fda34e9b9607495295339c5d"
format_stamp: "Formatted at 2026-08-08 18:32:12 on dist-test-slave-jztv"
server_key: "a214e27dcb4eae29251cd963a8518df3"
server_key_iv: "5b8b99585c0727d3d00cf802d4afdd9f"
server_key_version: "encryptionkey@0"
I20260808 18:32:12.607566 7612 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 18:32:12.610114 7637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:32:12.611408 7612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:32:12.611562 7612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "9e5c4bd8fda34e9b9607495295339c5d"
format_stamp: "Formatted at 2026-08-08 18:32:12 on dist-test-slave-jztv"
server_key: "a214e27dcb4eae29251cd963a8518df3"
server_key_iv: "5b8b99585c0727d3d00cf802d4afdd9f"
server_key_version: "encryptionkey@0"
I20260808 18:32:12.611686 7612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:32:12.629223 7612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:32:12.632264 7612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:32:12.632485 7612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:32:12.639899 7612 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:37341
I20260808 18:32:12.640196 7710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:37341 every 8 connection(s)
I20260808 18:32:12.640901 7612 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:32:12.643625 7712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:32:12.646062 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 7612
I20260808 18:32:12.646253 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:32:12.646521 7825 external_mini_cluster.cc:1428] Setting key 883ec857e16484030f36f349827ba7d9
I20260808 18:32:12.647433 7712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.164.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.125@KRBTEST.COM" created.
Aug 08 18:32:12 dist-test-slave-jztv krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213932, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:32:12.665128 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.649832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59968 (local address 127.7.164.126:37341)
0808 18:32:12.650358 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:12.650377 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:12.650606 (+ 229us) server_negotiation.cc:407] Connection header received
0808 18:32:12.651628 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:12.651633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:12.651873 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:12.652061 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:12.652889 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.653743 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.654761 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.655129 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.655295 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:12.659726 (+ 4431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:12.659735 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:12.659750 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:12.659781 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:12.661906 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:12.662580 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:12.662585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:12.662586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:12.662697 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:12.663085 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:12.663087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:12.663088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:12.663452 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:12.663559 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:12.664053 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 18:32:12.664268 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":326,"spinlock_wait_cycles":45952,"thread_start_us":162,"threads_started":1}
I20260808 18:32:12.665134 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.649832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37341 (local address 127.0.0.1:59968)
0808 18:32:12.650464 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.650500 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:12.651319 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:12.652060 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:12.652065 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:12.652502 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:12.652734 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:12.652747 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.653868 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.653873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:12.654642 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:12.654651 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.655284 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.655288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:12.655415 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:12.655952 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:12.655968 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:12.659544 (+ 3576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:12.662052 (+ 2508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:12.662064 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:12.662077 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:12.662445 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:12.662797 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:12.662802 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:12.662805 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:12.662979 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:12.663582 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:12.663592 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:12.663775 (+ 183us) client_negotiation.cc:769] Sending connection context
0808 18:32:12.663972 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 18:32:12.664366 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":51,"thread_start_us":162,"threads_started":1}
I20260808 18:32:12.667974 7712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:32:12.668798 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.668461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:37493 (local address 127.0.0.1:36528)
0808 18:32:12.668602 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.668696 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:37493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 18:32:12.669023 7643 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.164.125:37493: connect: Connection refused (error 111)
I20260808 18:32:12.669881 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.669645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:37493 (local address 127.0.0.1:36544)
0808 18:32:12.669767 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.669816 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:37493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 18:32:12.670068 7712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.125:37493: Network error: Client connection negotiation failed: client connection to 127.7.164.125:37493: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.125.keytab.
Entry for principal kudu/127.7.164.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.125.keytab.
I20260808 18:32:12.677742 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.125:37493
--webserver_interface=127.7.164.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.125.keytab
--principal=kudu/127.7.164.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:37341,127.7.164.125:37493,127.7.164.124:40391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:32:12.719419 7712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } attempt: 1
I20260808 18:32:12.720319 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.720001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:37493 (local address 127.0.0.1:36550)
0808 18:32:12.720171 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.720241 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:37493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 18:32:12.721119 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.720882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:37493 (local address 127.0.0.1:36566)
0808 18:32:12.720996 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.721049 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:37493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 18:32:12.721292 7712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.125:37493: Network error: Client connection negotiation failed: client connection to 127.7.164.125:37493: connect: Connection refused (error 111)
I20260808 18:32:12.789470 7712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } attempt: 2
I20260808 18:32:12.790361 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.789989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:37493 (local address 127.0.0.1:36568)
0808 18:32:12.790166 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.790270 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:37493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 18:32:12.791102 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.790879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:37493 (local address 127.0.0.1:36576)
0808 18:32:12.790994 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.791044 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:37493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 18:32:12.791250 7712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.125:37493: Network error: Client connection negotiation failed: client connection to 127.7.164.125:37493: connect: Connection refused (error 111)
W20260808 18:32:12.791592 7719 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:32:12.791884 7719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:32:12.791946 7719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:32:12.792024 7719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:32:12.796078 7719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:32:12.796154 7719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:32:12.796201 7719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:32:12.796257 7719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:32:12.796294 7719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:32:12.801674 7719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:37341,127.7.164.125:37493,127.7.164.124:40391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.125.keytab
--principal=kudu/127.7.164.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.125:37493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.164.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:32:12.802939 7719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:32:12.803996 7719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:32:12.812407 7731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:12.812492 7733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:12.812417 7730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:12.812497 7719 server_base.cc:1034] running on GCE node
I20260808 18:32:12.813268 7719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:32:12.814378 7719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:32:12.815591 7719 hybrid_clock.cc:648] HybridClock initialized: now 1786213932815567 us; error 41 us; skew 500 ppm
Aug 08 18:32:12 dist-test-slave-jztv krb5kdc[7596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213932, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:32:12.822499 7719 init.cc:377] Logged in from keytab as kudu/127.7.164.125@KRBTEST.COM (short username kudu)
I20260808 18:32:12.823416 7719 webserver.cc:460] Webserver started at http://127.7.164.125:45755/ using document root <none> and password file <none>
I20260808 18:32:12.823984 7719 fs_manager.cc:359] Metadata directory not provided
I20260808 18:32:12.824091 7719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:32:12.824333 7719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:32:12.826532 7719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/data/instance:
uuid: "ea4ec5df9280488cb964f04d0d0bba89"
format_stamp: "Formatted at 2026-08-08 18:32:12 on dist-test-slave-jztv"
server_key: "6b78754f0f9ab2398f90922dc6a4e5fd"
server_key_iv: "9a6aa018a3b84f04e4d4262103a22f59"
server_key_version: "encryptionkey@0"
I20260808 18:32:12.827134 7719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/wal/instance:
uuid: "ea4ec5df9280488cb964f04d0d0bba89"
format_stamp: "Formatted at 2026-08-08 18:32:12 on dist-test-slave-jztv"
server_key: "6b78754f0f9ab2398f90922dc6a4e5fd"
server_key_iv: "9a6aa018a3b84f04e4d4262103a22f59"
server_key_version: "encryptionkey@0"
I20260808 18:32:12.830812 7719 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:32:12.833220 7741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:32:12.834555 7719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:32:12.834663 7719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/wal
uuid: "ea4ec5df9280488cb964f04d0d0bba89"
format_stamp: "Formatted at 2026-08-08 18:32:12 on dist-test-slave-jztv"
server_key: "6b78754f0f9ab2398f90922dc6a4e5fd"
server_key_iv: "9a6aa018a3b84f04e4d4262103a22f59"
server_key_version: "encryptionkey@0"
I20260808 18:32:12.834746 7719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:32:12.865238 7719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:32:12.868304 7719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:32:12.868462 7719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:32:12.875813 7719 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.125:37493
I20260808 18:32:12.876122 7817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.125:37493 every 8 connection(s)
I20260808 18:32:12.876803 7719 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/data/info.pb
I20260808 18:32:12.879544 7818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:32:12.882536 7712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } attempt: 3
I20260808 18:32:12.883385 7818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:32:12.884992 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 7719
I20260808 18:32:12.885143 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-1/wal/instance
I20260808 18:32:12.885413 7825 external_mini_cluster.cc:1428] Setting key 41525f6525b09813a5bab807ec8ecfd7
Aug 08 18:32:12 dist-test-slave-jztv krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213932, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for kudu/127.7.164.125@KRBTEST.COM
Aug 08 18:32:12 dist-test-slave-jztv krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213932, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.125@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.164.124@KRBTEST.COM; defaulting to no policy
I20260808 18:32:12.896693 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.883121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:37493 (local address 127.0.0.1:36580)
0808 18:32:12.883290 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.883307 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:12.883410 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:12.885917 (+ 2507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:12.885923 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:12.886155 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:12.886343 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:12.886354 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.888251 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.888257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:12.889066 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:12.889076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.889774 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.889777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:12.889885 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:12.890474 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:12.890492 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:12.891822 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:12.894451 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:12.894455 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:12.894456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:12.894688 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:12.895083 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:12.895085 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:12.895087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:12.895166 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:12.896159 (+ 993us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:12.896162 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:12.896261 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 18:32:12.896374 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 18:32:12.896508 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 18:32:12.897364 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.885052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37341 (local address 127.0.0.1:59970)
0808 18:32:12.885448 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.885495 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:12.885668 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:12.886160 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:12.886167 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:12.886568 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:12.887003 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:12.887013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.888011 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.888015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:12.888814 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:12.888823 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.889439 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.889446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:12.889687 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:12.890334 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:12.890369 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:12.892535 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:12.894366 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:12.894378 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:12.894388 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:12.894714 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:12.895016 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:12.895019 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:12.895021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:12.895168 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:12.895633 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:12.895638 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:12.895925 (+ 287us) client_negotiation.cc:769] Sending connection context
0808 18:32:12.896123 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 18:32:12.896458 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":1,"thread_start_us":144,"threads_started":1}
I20260808 18:32:12.897374 7824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.883851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36580 (local address 127.7.164.125:37493)
0808 18:32:12.884281 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:12.884299 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:12.884345 (+ 46us) server_negotiation.cc:407] Connection header received
0808 18:32:12.885192 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:12.885217 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:12.885647 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:12.886529 (+ 882us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:12.886903 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.888114 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.889178 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.889646 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.889780 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:12.891994 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:12.892004 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:12.892021 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:12.892053 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:12.894331 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:12.894815 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:12.894824 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:12.894826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:12.894971 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:12.895685 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:12.895690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:12.895693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:12.896028 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:12.896173 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:12.896473 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 18:32:12.896608 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":285,"spinlock_wait_cycles":3328,"thread_start_us":145,"threads_started":1}
I20260808 18:32:12.898955 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.885070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59970 (local address 127.7.164.126:37341)
0808 18:32:12.885198 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:12.885205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:12.885581 (+ 376us) server_negotiation.cc:407] Connection header received
0808 18:32:12.885857 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:12.885860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:12.885922 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:12.885966 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:12.887155 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.887880 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.888948 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.889264 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.889360 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:12.892672 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:12.892676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:12.892678 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:12.892709 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:12.894200 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:12.894833 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:12.894836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:12.894838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:12.894917 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:12.895280 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:12.895285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:12.895288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:12.895524 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:12.895629 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:12.898689 (+ 3060us) server_negotiation.cc:299] Negotiation successful
0808 18:32:12.898836 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.124@KRBTEST.COM" created.
I20260808 18:32:12.899578 7712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:32:12.899964 7818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:32:12.900378 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.900058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59378)
0808 18:32:12.900236 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.900295 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260808 18:32:12.901197 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.900884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59392)
0808 18:32:12.901055 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.901118 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260808 18:32:12.901343 7712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:40391: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
Aug 08 18:32:12 dist-test-slave-jztv krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213932, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.125@KRBTEST.COM for kudu/127.7.164.125@KRBTEST.COM
I20260808 18:32:12.908981 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.900464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:37493 (local address 127.0.0.1:36596)
0808 18:32:12.900549 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.900564 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:12.900632 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:12.901897 (+ 1265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:12.901899 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:12.902062 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:12.902131 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:12.902136 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.902855 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.902858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:12.903344 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:12.903351 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.903834 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.903837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:12.903934 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:12.904390 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:12.904409 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:12.905819 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:12.907589 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:12.907593 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:12.907595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:12.907782 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:12.908064 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:12.908066 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:12.908068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:12.908148 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:12.908515 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:12.908518 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:12.908606 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 18:32:12.908721 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 18:32:12.908839 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 18:32:12.909014 7824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.901614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36596 (local address 127.7.164.125:37493)
0808 18:32:12.901717 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:12.901720 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:12.901731 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:32:12.901759 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:12.901760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:12.901791 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:12.901886 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:12.902256 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.902740 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.903458 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:12.903717 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:12.903837 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:12.905969 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:12.905973 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:12.905979 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:12.906004 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:12.907463 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:12.907886 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:12.907891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:12.907893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:12.907967 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:12.908234 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:12.908239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:12.908241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:12.908413 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:12.908515 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:12.908799 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 18:32:12.908897 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:32:12.909955 7818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:32:12.910692 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.910439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59400)
0808 18:32:12.910551 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.910631 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 18:32:12.910940 7747 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
I20260808 18:32:12.911890 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.911697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59410)
0808 18:32:12.911793 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.911836 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 18:32:12.912067 7818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:40391: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.124.keytab.
Entry for principal kudu/127.7.164.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.124.keytab.
I20260808 18:32:12.917761 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.124:40391
--webserver_interface=127.7.164.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.124.keytab
--principal=kudu/127.7.164.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:37341,127.7.164.125:37493,127.7.164.124:40391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:32:12.932505 7712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } attempt: 1
I20260808 18:32:12.933351 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.933031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59420)
0808 18:32:12.933190 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.933260 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 18:32:12.934188 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.933931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59436)
0808 18:32:12.934063 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.934116 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 18:32:12.934378 7712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:40391: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
I20260808 18:32:12.961349 7818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } attempt: 1
I20260808 18:32:12.962320 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.961954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59446)
0808 18:32:12.962133 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.962192 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 18:32:12.963058 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:12.962829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59458)
0808 18:32:12.962947 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:12.963002 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 18:32:12.963187 7818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:40391: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
I20260808 18:32:13.009603 7712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } attempt: 2
I20260808 18:32:13.010526 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.010151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59468)
0808 18:32:13.010351 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:13.010428 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 18:32:13.011320 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.011065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59480)
0808 18:32:13.011186 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:13.011250 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 18:32:13.011497 7712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:40391: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
I20260808 18:32:13.031359 7818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } attempt: 2
I20260808 18:32:13.032357 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.031971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59484)
0808 18:32:13.032197 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:13.032261 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260808 18:32:13.033226 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.032986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59488)
0808 18:32:13.033103 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:13.033159 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 18:32:13.033417 7818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:40391: Network error: Client connection negotiation failed: client connection to 127.7.164.124:40391: connect: Connection refused (error 111)
W20260808 18:32:13.037374 7840 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:32:13.037675 7840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:32:13.037729 7840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:32:13.037832 7840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:32:13.042084 7840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:32:13.042148 7840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:32:13.042168 7840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:32:13.042186 7840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:32:13.042253 7840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:32:13.047847 7840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:37341,127.7.164.125:37493,127.7.164.124:40391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.124.keytab
--principal=kudu/127.7.164.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.124:40391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.164.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:32:13.049031 7840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:32:13.050133 7840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:32:13.058945 7845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:13.058916 7847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:13.059144 7840 server_base.cc:1034] running on GCE node
W20260808 18:32:13.058929 7850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:13.059760 7840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:32:13.060832 7840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:32:13.062011 7840 hybrid_clock.cc:648] HybridClock initialized: now 1786213933061990 us; error 27 us; skew 500 ppm
Aug 08 18:32:13 dist-test-slave-jztv krb5kdc[7596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213933, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:32:13.064957 7840 init.cc:377] Logged in from keytab as kudu/127.7.164.124@KRBTEST.COM (short username kudu)
I20260808 18:32:13.065871 7840 webserver.cc:460] Webserver started at http://127.7.164.124:44817/ using document root <none> and password file <none>
I20260808 18:32:13.066519 7840 fs_manager.cc:359] Metadata directory not provided
I20260808 18:32:13.066614 7840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:32:13.066854 7840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:32:13.068940 7840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/data/instance:
uuid: "32f4d5408b3e4943831eae86b886da98"
format_stamp: "Formatted at 2026-08-08 18:32:13 on dist-test-slave-jztv"
server_key: "f01229296a48622d1cdcd915f4967471"
server_key_iv: "7fe8deb72afb91b4266c3904776149eb"
server_key_version: "encryptionkey@0"
I20260808 18:32:13.069523 7840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/wal/instance:
uuid: "32f4d5408b3e4943831eae86b886da98"
format_stamp: "Formatted at 2026-08-08 18:32:13 on dist-test-slave-jztv"
server_key: "f01229296a48622d1cdcd915f4967471"
server_key_iv: "7fe8deb72afb91b4266c3904776149eb"
server_key_version: "encryptionkey@0"
I20260808 18:32:13.073374 7840 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 18:32:13.075943 7863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:32:13.077313 7840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:32:13.077574 7840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/wal
uuid: "32f4d5408b3e4943831eae86b886da98"
format_stamp: "Formatted at 2026-08-08 18:32:13 on dist-test-slave-jztv"
server_key: "f01229296a48622d1cdcd915f4967471"
server_key_iv: "7fe8deb72afb91b4266c3904776149eb"
server_key_version: "encryptionkey@0"
I20260808 18:32:13.077704 7840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:32:13.091082 7840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:32:13.094115 7840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:32:13.094375 7840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:32:13.101542 7840 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.124:40391
I20260808 18:32:13.101852 7936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.124:40391 every 8 connection(s)
I20260808 18:32:13.102605 7840 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/data/info.pb
I20260808 18:32:13.105152 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 7840
I20260808 18:32:13.105254 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213640155572-7825-0/minicluster-data/master-2/wal/instance
I20260808 18:32:13.105554 7825 external_mini_cluster.cc:1428] Setting key da3803034062480736f6f33fdebc5e5b
I20260808 18:32:13.105626 7937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:32:13.110705 7712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } attempt: 3
I20260808 18:32:13.111016 7937 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:32:13 dist-test-slave-jztv krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213932, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for kudu/127.7.164.124@KRBTEST.COM
Aug 08 18:32:13 dist-test-slave-jztv krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.124@KRBTEST.COM
Aug 08 18:32:13 dist-test-slave-jztv krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213933, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.124@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:32:13.123016 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.111234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59498)
0808 18:32:13.111366 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:13.111382 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:13.111500 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:13.111934 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:13.111940 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:13.112175 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:13.112312 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:13.112321 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.113554 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.113557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:13.114362 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:13.114374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.115083 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.115087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:13.115179 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.115693 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:13.115708 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:13.117405 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:13.120514 (+ 3109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:13.120517 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:13.120519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:13.120810 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:13.121242 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:13.121246 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:13.121249 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:13.121384 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:13.122200 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:13.122205 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:13.122615 (+ 410us) client_negotiation.cc:769] Sending connection context
0808 18:32:13.122708 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 18:32:13.122831 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 18:32:13.124653 7818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } attempt: 3
I20260808 18:32:13.124841 7941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.107017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.7.164.124:40391)
0808 18:32:13.107434 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.107454 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.107499 (+ 45us) server_negotiation.cc:407] Connection header received
0808 18:32:13.108551 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.108578 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.109017 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.109418 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.110334 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.111871 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.114306 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.114937 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.115117 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.118390 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:13.118426 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:13.118430 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:13.118469 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:13.120844 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.121394 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.121398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.121400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.121489 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.121880 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.121884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.121887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.122133 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:13.122423 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:13.123385 (+ 962us) server_negotiation.cc:299] Negotiation successful
0808 18:32:13.123878 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":180,"server-negotiator.queue_time_us":267,"thread_start_us":104,"threads_started":1}
I20260808 18:32:13.124874 7945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.111276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59498 (local address 127.7.164.124:40391)
0808 18:32:13.111656 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.111667 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.111682 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:32:13.111721 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.111726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.111771 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.112464 (+ 693us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.112585 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.113403 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.114520 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.114938 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.115422 (+ 484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.117607 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:13.117614 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:13.117626 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:13.117658 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:13.120368 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.120953 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.120959 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.120965 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.121130 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.121513 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.121519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.121523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.122035 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:13.123484 (+ 1449us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:13.123562 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 18:32:13.123790 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":282,"thread_start_us":143,"threads_started":1}
I20260808 18:32:13.125489 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.113611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59984 (local address 127.7.164.126:37341)
0808 18:32:13.113759 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.113763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.114101 (+ 338us) server_negotiation.cc:407] Connection header received
0808 18:32:13.114271 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.114278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.114331 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.114418 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.115344 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.116096 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.117214 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.117446 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.117577 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.121146 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:13.121152 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:13.121158 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:13.121190 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:13.123334 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.123947 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.123950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.123952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.124042 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.124362 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.124366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.124367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.124644 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:13.124743 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:13.125209 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 18:32:13.125369 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 18:32:13.125957 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.113550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:37341 (local address 127.0.0.1:59984)
0808 18:32:13.113947 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:13.114002 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:13.114137 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:13.114502 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:13.114509 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:13.115036 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:13.115203 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:13.115210 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.116260 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.116264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:13.117014 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:13.117046 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.117687 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.117693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:13.117845 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.118535 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:13.118548 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:13.120986 (+ 2438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:13.123465 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:13.123473 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:13.123479 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:13.123833 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:13.124147 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:13.124150 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:13.124152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:13.124252 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:13.124798 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:13.124804 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:13.124937 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 18:32:13.125637 (+ 700us) client_negotiation.cc:240] Negotiation successful
0808 18:32:13.125807 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":119,"threads_started":1}
I20260808 18:32:13.128170 7937 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:32:13.129899 7712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d: Bootstrap starting.
Aug 08 18:32:13 dist-test-slave-jztv krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213932, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.125@KRBTEST.COM for kudu/127.7.164.124@KRBTEST.COM
I20260808 18:32:13.133107 7712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d: Neither blocks nor log segments found. Creating new log.
Aug 08 18:32:13 dist-test-slave-jztv krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
Aug 08 18:32:13 dist-test-slave-jztv krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213933, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.124@KRBTEST.COM for kudu/127.7.164.125@KRBTEST.COM
I20260808 18:32:13.134303 7712 log.cc:824] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d: Log is configured to *not* fsync() on all Append() calls
I20260808 18:32:13.136523 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.125323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59514)
0808 18:32:13.125750 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:13.125766 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:13.125850 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:13.126603 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:13.126606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:13.126793 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:13.126922 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:13.126935 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.128011 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.128014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:13.128821 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:13.128834 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.129437 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.129443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:13.129532 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.130064 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:13.130079 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:13.131879 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:13.134252 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:13.134256 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:13.134258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:13.134803 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:13.135158 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:13.135164 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:13.135167 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:13.135275 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:13.135730 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:13.135936 (+ 206us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:13.136027 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 18:32:13.136175 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 18:32:13.136325 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344}
I20260808 18:32:13.136854 7941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.126006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59514 (local address 127.7.164.124:40391)
0808 18:32:13.126366 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.126370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.126383 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:32:13.126415 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.126418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.126457 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.127040 (+ 583us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.127132 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.127875 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.128976 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.129286 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.132014 (+ 2728us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.132122 (+ 108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:13.132125 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:13.132128 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:13.132144 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:13.134100 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.134931 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.134933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.134934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.135045 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.135413 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.135419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.135424 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.135617 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:13.135742 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:13.136580 (+ 838us) server_negotiation.cc:299] Negotiation successful
0808 18:32:13.136705 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293}
I20260808 18:32:13.137991 7712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d: No bootstrap required, opened a new log
I20260808 18:32:13.138422 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.127667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59992 (local address 127.7.164.126:37341)
0808 18:32:13.127774 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.127777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.127787 (+ 10us) server_negotiation.cc:407] Connection header received
0808 18:32:13.127814 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.127816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.127846 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.127936 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.128750 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.129446 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.130583 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.130917 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.131020 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.134126 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:13.134130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:13.134132 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:13.134157 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:13.135988 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.136525 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.136529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.136531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.136617 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.136948 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.136951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.136953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.137170 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:13.137325 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:13.138116 (+ 791us) server_negotiation.cc:299] Negotiation successful
0808 18:32:13.138276 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:32:13.139566 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.128700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:37493 (local address 127.0.0.1:36602)
0808 18:32:13.129120 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:13.129133 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:13.129233 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:13.129552 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:13.129557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:13.129733 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:13.129837 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:13.129844 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.130866 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.130869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:13.131732 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:13.131745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.132557 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.132560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:13.132684 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.133145 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:13.133160 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:13.134634 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:13.137788 (+ 3154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:13.137791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:13.137793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:13.138051 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:13.138418 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:13.138421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:13.138423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:13.138506 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:13.139059 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:13.139062 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:13.139158 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 18:32:13.139259 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 18:32:13.139425 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360}
I20260808 18:32:13.140151 7824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.128923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36602 (local address 127.7.164.125:37493)
0808 18:32:13.129030 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.129035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.129180 (+ 145us) server_negotiation.cc:407] Connection header received
0808 18:32:13.129323 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.129326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.129364 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.129407 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.129967 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.130683 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.131874 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.132283 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.132377 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.134792 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:13.134798 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:13.134801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:13.134832 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:13.137460 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.138172 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.138179 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.138183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.138309 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.138602 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.138606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.138607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.138886 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:13.138986 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:13.139893 (+ 907us) server_negotiation.cc:299] Negotiation successful
0808 18:32:13.140011 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:32:13.140460 7818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89: Bootstrap starting.
I20260808 18:32:13.142617 7712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } }
I20260808 18:32:13.142868 7937 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:32:13.142947 7712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5c4bd8fda34e9b9607495295339c5d, State: Initialized, Role: FOLLOWER
I20260808 18:32:13.143344 7712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } }
I20260808 18:32:13.143463 7818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89: Neither blocks nor log segments found. Creating new log.
I20260808 18:32:13.144299 7818 log.cc:824] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89: Log is configured to *not* fsync() on all Append() calls
I20260808 18:32:13.145009 7712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 18:32:13 dist-test-slave-jztv krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.125@KRBTEST.COM
I20260808 18:32:13.146911 7818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89: No bootstrap required, opened a new log
Aug 08 18:32:13 dist-test-slave-jztv krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213933, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.124@KRBTEST.COM for kudu/127.7.164.124@KRBTEST.COM
I20260808 18:32:13.152251 7818 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } }
W20260808 18:32:13.152437 7968 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:32:13.152521 7968 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:32:13.152658 7818 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea4ec5df9280488cb964f04d0d0bba89, State: Initialized, Role: FOLLOWER
I20260808 18:32:13.152940 7953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } } }
I20260808 18:32:13.153142 7818 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } }
I20260808 18:32:13.153410 7953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:32:13.153517 7824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.139933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36604 (local address 127.7.164.125:37493)
0808 18:32:13.140298 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.140302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.140315 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:32:13.140420 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.140423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.140477 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.140525 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.141381 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.142144 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.143765 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.144070 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.144148 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.147522 (+ 3374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:13.147526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:13.147528 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:13.147559 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:13.151004 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.151630 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.151634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.151635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.151735 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.152100 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.152103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.152105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.152341 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:13.152543 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:13.152994 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 18:32:13.153330 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":124,"threads_started":1}
I20260808 18:32:13.154640 7818 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:32:13.150929 7612 master_runner.cc:186] Error getting master registration for 127.7.164.125:37493: OK, instance_id { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" instance_seqno: 1786213932873723 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:32:13.154937 7612 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:32:13.157609 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:32:13.158985 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.143382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:40391 (local address 127.0.0.1:59530)
0808 18:32:13.143652 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:13.143667 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:13.143750 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:13.145086 (+ 1336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:13.145092 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:13.145285 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:13.145409 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:13.145418 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.146421 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.146424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:13.147147 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:13.147159 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.147703 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.147707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:13.147778 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.148337 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:13.148362 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:13.149967 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:13.157201 (+ 7234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:13.157204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:13.157206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:13.157515 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:13.157837 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:13.157840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:13.157842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:13.157939 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:13.158472 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:13.158475 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:13.158567 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 18:32:13.158657 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 18:32:13.158817 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260808 18:32:13.159225 7676 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9e5c4bd8fda34e9b9607495295339c5d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59992
I20260808 18:32:13.159611 7676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 18:32:13.159761 7676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:32:13.159786 7941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.143648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59530 (local address 127.7.164.124:40391)
0808 18:32:13.144852 (+ 1204us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.144857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.144872 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:32:13.144915 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.144917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.144964 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.145522 (+ 558us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.145597 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.146297 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.147275 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.147585 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.147987 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.155037 (+ 7050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:13.155049 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:13.155053 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:13.155087 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:13.157074 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.157650 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.157655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.157658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.157742 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.158043 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.158045 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.158047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.158367 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:13.159442 (+ 1075us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:13.159526 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 18:32:13.159647 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1116}
I20260808 18:32:13.161271 7676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } }
I20260808 18:32:13.162899 7937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98: Bootstrap starting.
I20260808 18:32:13.162989 7782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e5c4bd8fda34e9b9607495295339c5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ea4ec5df9280488cb964f04d0d0bba89"
I20260808 18:32:13.163251 7782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89 [term 0 FOLLOWER]: Advancing to term 1
W20260808 18:32:13.163755 7985 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:32:13.163836 7985 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:32:13.165012 7782 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5c4bd8fda34e9b9607495295339c5d in term 1.
I20260808 18:32:13.165570 7643 leader_election.cc:302] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e5c4bd8fda34e9b9607495295339c5d, ea4ec5df9280488cb964f04d0d0bba89; no voters:
I20260808 18:32:13.165688 7937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98: Neither blocks nor log segments found. Creating new log.
I20260808 18:32:13.166059 7953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:32:13.166352 7953 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [term 1 LEADER]: Becoming Leader. State: Replica: 9e5c4bd8fda34e9b9607495295339c5d, State: Running, Role: LEADER
I20260808 18:32:13.166604 7937 log.cc:824] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98: Log is configured to *not* fsync() on all Append() calls
I20260808 18:32:13.166708 7953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } }
I20260808 18:32:13.167436 7676 leader_election.cc:288] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [CANDIDATE]: Term 1 election: Requested vote from peers ea4ec5df9280488cb964f04d0d0bba89 (127.7.164.125:37493), 32f4d5408b3e4943831eae86b886da98 (127.7.164.124:40391)
I20260808 18:32:13.168777 7953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e5c4bd8fda34e9b9607495295339c5d. Latest consensus state: current_term: 1 leader_uuid: "9e5c4bd8fda34e9b9607495295339c5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } } }
I20260808 18:32:13.168892 7953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:13.170480 7971 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } } }
I20260808 18:32:13.170599 7971 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:32:13.167616 7719 master_runner.cc:186] Error getting master registration for 127.7.164.124:40391: OK, instance_id { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" instance_seqno: 1786213933099617 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:32:13.171192 7719 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:32:13.173015 7992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:32:13.174504 7992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:32:13.183862 7937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98: No bootstrap required, opened a new log
I20260808 18:32:13.186291 7824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.171076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36612 (local address 127.7.164.125:37493)
0808 18:32:13.171197 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.171203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.173085 (+ 1882us) server_negotiation.cc:407] Connection header received
0808 18:32:13.173467 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.173472 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.173542 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.173606 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.175355 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.176151 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.178011 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.178398 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.178483 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.181512 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:13.181517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:13.181520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:13.181550 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:13.183651 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.184448 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.184452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.184454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.184548 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.185034 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.185045 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.185047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.185279 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:13.185405 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:13.185963 (+ 558us) server_negotiation.cc:299] Negotiation successful
0808 18:32:13.186097 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 18:32:13.189396 7937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } }
I20260808 18:32:13.189759 7937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32f4d5408b3e4943831eae86b886da98, State: Initialized, Role: FOLLOWER
I20260808 18:32:13.190203 7937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } }
I20260808 18:32:13.190953 8005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } } }
I20260808 18:32:13.191239 8005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:32:13.192097 7937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:32:13.193848 7905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e5c4bd8fda34e9b9607495295339c5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "32f4d5408b3e4943831eae86b886da98"
I20260808 18:32:13.194093 7905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:32:13.195575 7905 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5c4bd8fda34e9b9607495295339c5d in term 1.
I20260808 18:32:13.195915 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.172426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60006 (local address 127.7.164.126:37341)
0808 18:32:13.172540 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.172545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.176144 (+ 3599us) server_negotiation.cc:407] Connection header received
0808 18:32:13.176339 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.176343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.176393 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.176444 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.182245 (+ 5801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.183030 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.185576 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.185953 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.186042 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.189893 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:13.189898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:13.189901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:13.189936 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:13.193624 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.194241 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.194245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.194248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.194365 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.194729 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.194735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.194738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.194992 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:13.195142 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:13.195565 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 18:32:13.195717 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 18:32:13.198345 7941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:13.171981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59546 (local address 127.7.164.124:40391)
0808 18:32:13.172098 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.172104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.173383 (+ 1279us) server_negotiation.cc:407] Connection header received
0808 18:32:13.174795 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.174799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.174853 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.174908 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.177433 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.178200 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.184392 (+ 6192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:13.184747 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:13.184839 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:13.190338 (+ 5499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:13.190342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:13.190346 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:13.190380 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:13.192841 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.196653 (+ 3812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.196657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.196659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.196754 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:13.197060 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:13.197063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:13.197065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:13.197277 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:13.197487 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:13.197899 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 18:32:13.198052 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 18:32:13.201630 7782 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89 [term 1 FOLLOWER]: Refusing update from remote peer 9e5c4bd8fda34e9b9607495295339c5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:32:13.202467 7953 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:32:13.203655 7905 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98 [term 1 FOLLOWER]: Refusing update from remote peer 9e5c4bd8fda34e9b9607495295339c5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:32:13.206422 7953 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:32:13.211086 7971 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e5c4bd8fda34e9b9607495295339c5d. Latest consensus state: current_term: 1 leader_uuid: "9e5c4bd8fda34e9b9607495295339c5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } } }
I20260808 18:32:13.211233 7971 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:32:13.212546 8005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e5c4bd8fda34e9b9607495295339c5d. Latest consensus state: current_term: 1 leader_uuid: "9e5c4bd8fda34e9b9607495295339c5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } } }
I20260808 18:32:13.212728 8005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:32:13.214993 7953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e5c4bd8fda34e9b9607495295339c5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } } }
I20260808 18:32:13.215142 7953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:13.215991 8003 mvcc.cc:205] Tried to move back new op lower bound from 7316332270384168960 to 7316332270255271936. Current Snapshot: MvccSnapshot[applied={T|T < 7316332270384168960}]
I20260808 18:32:13.217075 8005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e5c4bd8fda34e9b9607495295339c5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } } }
I20260808 18:32:13.217288 8005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:32:13.220513 7988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e5c4bd8fda34e9b9607495295339c5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } } }
I20260808 18:32:13.220674 7988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d [sys.catalog]: This master's current role is: LEADER
W20260808 18:32:13.221468 8018 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:32:13.221776 8018 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:32:13.221859 7971 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e5c4bd8fda34e9b9607495295339c5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5c4bd8fda34e9b9607495295339c5d" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 37341 } } peers { permanent_uuid: "ea4ec5df9280488cb964f04d0d0bba89" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 37493 } } peers { permanent_uuid: "32f4d5408b3e4943831eae86b886da98" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 40391 } } }
I20260808 18:32:13.222036 7971 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:32:13.224372 8021 mvcc.cc:205] Tried to move back new op lower bound from 7316332270384168960 to 7316332270255271936. Current Snapshot: MvccSnapshot[applied={T|T < 7316332270384168960}]
I20260808 18:32:13.226107 7992 catalog_manager.cc:1347] Generated new cluster ID: 4b70a702700b4d9ea3281a1509773a1a
I20260808 18:32:13.226181 7992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:32:13.224586 8024 mvcc.cc:205] Tried to move back new op lower bound from 7316332270384168960 to 7316332270255271936. Current Snapshot: MvccSnapshot[applied={T|T < 7316332270384168960}]
I20260808 18:32:13.264690 7992 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:32:13.268884 7992 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:32:13.290750 7992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e5c4bd8fda34e9b9607495295339c5d: Generated new TSK 0
I20260808 18:32:13.291781 7992 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:32:13.327278 7824 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:13.323701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36618 (local address 127.7.164.125:37493)
0808 18:32:13.323840 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.323845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.325927 (+ 2082us) server_negotiation.cc:407] Connection header received
0808 18:32:13.326181 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.326186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.326279 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.326346 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.327108 (+ 762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36618: BlockingRecv error: recv got EOF from 127.0.0.1:36618 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 18:32:13.328923 7941 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:13.324528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.7.164.124:40391)
0808 18:32:13.324679 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.324684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.327839 (+ 3155us) server_negotiation.cc:407] Connection header received
0808 18:32:13.328052 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.328057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.328130 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.328191 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.328788 (+ 597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59554: BlockingRecv error: recv got EOF from 127.0.0.1:59554 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 18:32:13.330344 7716 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:13.325241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60018 (local address 127.7.164.126:37341)
0808 18:32:13.325364 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.325369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.329344 (+ 3975us) server_negotiation.cc:407] Connection header received
0808 18:32:13.329533 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.329536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.329591 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.329643 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.330207 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60018: BlockingRecv error: recv got EOF from 127.0.0.1:60018 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 18:32:13.335237 7824 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:13.333616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36628 (local address 127.7.164.125:37493)
0808 18:32:13.333721 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.333725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.334277 (+ 552us) server_negotiation.cc:407] Connection header received
0808 18:32:13.334500 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.334503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.334577 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.334629 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.335134 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36628: BlockingRecv error: recv got EOF from 127.0.0.1:36628 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 18:32:13.340255 7824 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:13.338729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36636 (local address 127.7.164.125:37493)
0808 18:32:13.338838 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.338842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.339366 (+ 524us) server_negotiation.cc:407] Connection header received
0808 18:32:13.339553 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.339556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.339602 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.339651 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.340151 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36636: BlockingRecv error: recv got EOF from 127.0.0.1:36636 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 18:32:13.345705 7824 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:13.344220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36648 (local address 127.7.164.125:37493)
0808 18:32:13.344318 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.344321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.344841 (+ 520us) server_negotiation.cc:407] Connection header received
0808 18:32:13.345031 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.345034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.345079 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.345128 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.345608 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36648: BlockingRecv error: recv got EOF from 127.0.0.1:36648 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 18:32:13.353214 7824 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:13.351731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36662 (local address 127.7.164.125:37493)
0808 18:32:13.351865 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.351869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.352365 (+ 496us) server_negotiation.cc:407] Connection header received
0808 18:32:13.352539 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.352543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.352588 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.352637 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.353115 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36662: BlockingRecv error: recv got EOF from 127.0.0.1:36662 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 18:32:13.365307 7824 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:13.363614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36672 (local address 127.7.164.125:37493)
0808 18:32:13.363735 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.363740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.364301 (+ 561us) server_negotiation.cc:407] Connection header received
0808 18:32:13.364517 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.364521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.364576 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.364630 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.365195 (+ 565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36672: BlockingRecv error: recv got EOF from 127.0.0.1:36672 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 18:32:13.387403 7824 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:13.385377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.7.164.125:37493)
0808 18:32:13.385513 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.385520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.386275 (+ 755us) server_negotiation.cc:407] Connection header received
0808 18:32:13.386517 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.386521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.386584 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.386642 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.387279 (+ 637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36680: BlockingRecv error: recv got EOF from 127.0.0.1:36680 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 18:32:13.423455 7824 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:13.422238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36686 (local address 127.7.164.125:37493)
0808 18:32:13.422372 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.422377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.422540 (+ 163us) server_negotiation.cc:407] Connection header received
0808 18:32:13.422664 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.422671 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.422738 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.422833 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.423325 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36686: BlockingRecv error: recv got EOF from 127.0.0.1:36686 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 18:32:13.490983 7824 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:13.489682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36700 (local address 127.7.164.125:37493)
0808 18:32:13.489872 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.489877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.490019 (+ 142us) server_negotiation.cc:407] Connection header received
0808 18:32:13.490207 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.490239 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.490289 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.490391 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.490875 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36700: BlockingRecv error: recv got EOF from 127.0.0.1:36700 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260808 18:32:13.622599 7824 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:13.621432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36704 (local address 127.7.164.125:37493)
0808 18:32:13.621643 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.621648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.621746 (+ 98us) server_negotiation.cc:407] Connection header received
0808 18:32:13.621879 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.621881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.621932 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.622012 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.622473 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36704: BlockingRecv error: recv got EOF from 127.0.0.1:36704 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
W20260808 18:32:13.881832 7824 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:13.880809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36712 (local address 127.7.164.125:37493)
0808 18:32:13.880934 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:13.880938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:13.881039 (+ 101us) server_negotiation.cc:407] Connection header received
0808 18:32:13.881150 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:13.881154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:13.881209 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:13.881305 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:13.881734 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36712: BlockingRecv error: recv got EOF from 127.0.0.1:36712 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:32:14.165208 7985 catalog_manager.cc:1259] Loaded cluster ID: 4b70a702700b4d9ea3281a1509773a1a
I20260808 18:32:14.165297 7985 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89: loading cluster ID for follower catalog manager: success
I20260808 18:32:14.166908 7985 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89: acquiring CA information for follower catalog manager: success
I20260808 18:32:14.167995 7985 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ea4ec5df9280488cb964f04d0d0bba89: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:32:14.227737 8018 catalog_manager.cc:1259] Loaded cluster ID: 4b70a702700b4d9ea3281a1509773a1a
I20260808 18:32:14.227828 8018 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98: loading cluster ID for follower catalog manager: success
I20260808 18:32:14.229471 8018 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98: acquiring CA information for follower catalog manager: success
I20260808 18:32:14.230618 8018 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 32f4d5408b3e4943831eae86b886da98: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:32:14.400270 8122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:14.396084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36724 (local address 127.7.164.125:37493)
0808 18:32:14.396341 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:14.396347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:14.396446 (+ 99us) server_negotiation.cc:407] Connection header received
0808 18:32:14.396626 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:14.396629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:14.396766 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:14.396886 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:32:14.397173 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:14.397719 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:14.398698 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:14.398927 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:14.399035 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:14.399391 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:32:14.399849 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:32:14.399984 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:14.400047 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 18:32:14.400108 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":68,"threads_started":1}
I20260808 18:32:14.407083 8128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:14.403561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59556 (local address 127.7.164.124:40391)
0808 18:32:14.403811 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:14.403815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:14.403829 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:32:14.403911 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:14.403913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:14.404027 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:14.404130 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:32:14.404320 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:14.404831 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:14.405602 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:14.405815 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:14.405891 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:14.406148 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:32:14.406660 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:32:14.406783 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:14.406894 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 18:32:14.406955 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":107,"threads_started":1}
I20260808 18:32:14.414089 8122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:14.409944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36736 (local address 127.7.164.125:37493)
0808 18:32:14.410074 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:14.410079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:14.410437 (+ 358us) server_negotiation.cc:407] Connection header received
0808 18:32:14.410488 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:14.410490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:14.410531 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:14.410624 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:32:14.410853 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:14.411576 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:14.412682 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:14.413036 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:14.413137 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:14.413491 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:32:14.413605 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:32:14.413681 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:14.413843 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 18:32:14.413909 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:32:14.414604 8128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:14.410366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.7.164.124:40391)
0808 18:32:14.410483 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:14.410487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:14.410698 (+ 211us) server_negotiation.cc:407] Connection header received
0808 18:32:14.410871 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:14.410874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:14.410931 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:14.410976 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:32:14.411411 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:14.412144 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:14.413298 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:14.413642 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:14.413747 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:14.414080 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:32:14.414150 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:32:14.414244 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:14.414410 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 18:32:14.414470 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:32:14.414911 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:14.409926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60022 (local address 127.7.164.126:37341)
0808 18:32:14.410546 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:14.410550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:14.410565 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:32:14.410665 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:14.410671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:14.410845 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:14.410962 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:32:14.411221 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:14.412063 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:14.413136 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:14.413463 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:14.413566 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:14.413909 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:32:14.414289 (+ 380us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:32:14.414660 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:14.414700 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 18:32:14.414765 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":63,"threads_started":1}
I20260808 18:32:14.418520 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 7612
I20260808 18:32:14.435638 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 7719
I20260808 18:32:14.447613 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 7840
2026-08-08T18:32:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3358 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:32:14 dist-test-slave-jztv krb5kdc[8143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:32:14 dist-test-slave-jztv krb5kdc[8143](info): set up 2 sockets
Aug 08 18:32:14 dist-test-slave-jztv krb5kdc[8143](info): commencing operation
krb5kdc: starting...
W20260808 18:32:15.682335 7825 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/test-user.keytab.
Aug 08 18:32:15 dist-test-slave-jztv krb5kdc[8143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:32:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:32:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.164.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
Entry for principal kudu/127.7.164.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab.
I20260808 18:32:15.819051 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:33651
--webserver_interface=127.7.164.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:37205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:33651,127.7.164.125:39649,127.7.164.124:44965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:32:15.935427 8161 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:32:15.935758 8161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:32:15.935808 8161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:32:15.935889 8161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:32:15.939949 8161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:32:15.940011 8161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:32:15.940030 8161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:32:15.940047 8161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:32:15.940062 8161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:32:15.945293 8161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:37205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:33651,127.7.164.125:39649,127.7.164.124:44965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.126.keytab
--principal=kudu/127.7.164.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.126:33651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.164.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:32:15.946451 8161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:32:15.947440 8161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:32:15.955950 8166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:15.955996 8169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:15.956133 8161 server_base.cc:1034] running on GCE node
W20260808 18:32:15.955950 8167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:15.956909 8161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:32:15.958037 8161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:32:15.959201 8161 hybrid_clock.cc:648] HybridClock initialized: now 1786213935959176 us; error 40 us; skew 500 ppm
Aug 08 18:32:15 dist-test-slave-jztv krb5kdc[8143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213935, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:32:15.962023 8161 init.cc:377] Logged in from keytab as kudu/127.7.164.126@KRBTEST.COM (short username kudu)
I20260808 18:32:15.962988 8161 webserver.cc:460] Webserver started at http://127.7.164.126:41099/ using document root <none> and password file <none>
I20260808 18:32:15.963570 8161 fs_manager.cc:359] Metadata directory not provided
I20260808 18:32:15.963667 8161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:32:15.963913 8161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:32:15.966013 8161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/data/instance:
uuid: "0278df41af8549499b0b1b50dbd476b7"
format_stamp: "Formatted at 2026-08-08 18:32:15 on dist-test-slave-jztv"
server_key: "fdebfd8618f592512031ed9071b4e9b1"
server_key_iv: "5fc082d00af8203819bca190aa3c6bc7"
server_key_version: "encryptionkey@0"
I20260808 18:32:15.966668 8161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/wal/instance:
uuid: "0278df41af8549499b0b1b50dbd476b7"
format_stamp: "Formatted at 2026-08-08 18:32:15 on dist-test-slave-jztv"
server_key: "fdebfd8618f592512031ed9071b4e9b1"
server_key_iv: "5fc082d00af8203819bca190aa3c6bc7"
server_key_version: "encryptionkey@0"
I20260808 18:32:15.970424 8161 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:32:15.972909 8184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:32:15.974141 8161 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:32:15.974316 8161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/wal
uuid: "0278df41af8549499b0b1b50dbd476b7"
format_stamp: "Formatted at 2026-08-08 18:32:15 on dist-test-slave-jztv"
server_key: "fdebfd8618f592512031ed9071b4e9b1"
server_key_iv: "5fc082d00af8203819bca190aa3c6bc7"
server_key_version: "encryptionkey@0"
I20260808 18:32:15.974437 8161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:32:15.987954 8161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:32:15.991226 8161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:32:15.991434 8161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:32:15.998562 8161 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.126:33651
I20260808 18:32:15.998832 8264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.126:33651 every 8 connection(s)
I20260808 18:32:15.999552 8161 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/data/info.pb
I20260808 18:32:16.002734 8266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:32:16.006537 8266 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:32:16.007251 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 8161
I20260808 18:32:16.007395 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-0/wal/instance
I20260808 18:32:16.007714 7825 external_mini_cluster.cc:1428] Setting key d7c1d7ac32dfb87b0a1bc7ba5b9ec39b
Aug 08 18:32:16 dist-test-slave-jztv krb5kdc[8143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213935, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.164.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.125@KRBTEST.COM" created.
I20260808 18:32:16.023164 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.009112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58320 (local address 127.7.164.126:33651)
0808 18:32:16.009675 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.009692 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.009875 (+ 183us) server_negotiation.cc:407] Connection header received
0808 18:32:16.010952 (+ 1077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.010956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.011143 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.011330 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.012333 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.013199 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.014865 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.015229 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.015343 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.018070 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.018077 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.018087 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.018107 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.019688 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.020234 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.020245 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.020246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.020347 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.020705 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.020710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.020712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.021126 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.021265 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.021721 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.022068 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":444,"spinlock_wait_cycles":26496,"thread_start_us":153,"threads_started":1}
I20260808 18:32:16.023164 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.009112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33651 (local address 127.0.0.1:58320)
0808 18:32:16.009691 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.009753 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:16.010656 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:16.011327 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:16.011333 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:16.011802 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:16.012165 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.012187 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.013375 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.013382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.014729 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.014745 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.015396 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.015402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.015565 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.016168 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:16.016193 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:16.017920 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:16.019798 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.019809 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.019819 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.020123 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.020448 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.020451 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.020452 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.020609 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.021239 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:16.021245 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:16.021394 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 18:32:16.021654 (+ 260us) client_negotiation.cc:240] Negotiation successful
0808 18:32:16.022115 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":165,"thread_start_us":141,"threads_started":1}
I20260808 18:32:16.025646 8266 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:32:16.026639 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.026248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:39649 (local address 127.0.0.1:51914)
0808 18:32:16.026413 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.026526 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:39649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 18:32:16.026870 8192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.164.125:39649: connect: Connection refused (error 111)
I20260808 18:32:16.027829 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.027565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:39649 (local address 127.0.0.1:51924)
0808 18:32:16.027689 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.027749 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:39649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 18:32:16.027988 8266 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.125:39649: Network error: Client connection negotiation failed: client connection to 127.7.164.125:39649: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.125.keytab.
Entry for principal kudu/127.7.164.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.125.keytab.
I20260808 18:32:16.037330 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.125:39649
--webserver_interface=127.7.164.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.125.keytab
--principal=kudu/127.7.164.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:37205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:33651,127.7.164.125:39649,127.7.164.124:44965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:32:16.077261 8266 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } attempt: 1
I20260808 18:32:16.078200 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.077857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:39649 (local address 127.0.0.1:51934)
0808 18:32:16.078031 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.078105 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:39649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 18:32:16.079012 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.078768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:39649 (local address 127.0.0.1:51942)
0808 18:32:16.078885 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.078940 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:39649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 18:32:16.079170 8266 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.125:39649: Network error: Client connection negotiation failed: client connection to 127.7.164.125:39649: connect: Connection refused (error 111)
I20260808 18:32:16.147377 8266 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } attempt: 2
I20260808 18:32:16.148283 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.147932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:39649 (local address 127.0.0.1:51944)
0808 18:32:16.148110 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.148185 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:39649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 18:32:16.149086 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.148817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:39649 (local address 127.0.0.1:51952)
0808 18:32:16.148936 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.149003 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.125:39649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 18:32:16.149266 8266 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.125:39649: Network error: Client connection negotiation failed: client connection to 127.7.164.125:39649: connect: Connection refused (error 111)
W20260808 18:32:16.152047 8273 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:32:16.152311 8273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:32:16.152385 8273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:32:16.152458 8273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:32:16.156607 8273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:32:16.156664 8273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:32:16.156683 8273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:32:16.156728 8273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:32:16.156759 8273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:32:16.162186 8273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:37205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:33651,127.7.164.125:39649,127.7.164.124:44965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.125.keytab
--principal=kudu/127.7.164.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.125:39649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.164.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:32:16.163437 8273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:32:16.164479 8273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:32:16.173020 8273 server_base.cc:1034] running on GCE node
W20260808 18:32:16.172987 8279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:16.172966 8282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:16.172989 8278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:16.173815 8273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:32:16.174942 8273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:32:16.176134 8273 hybrid_clock.cc:648] HybridClock initialized: now 1786213936176114 us; error 35 us; skew 500 ppm
Aug 08 18:32:16 dist-test-slave-jztv krb5kdc[8143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213936, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:32:16.179001 8273 init.cc:377] Logged in from keytab as kudu/127.7.164.125@KRBTEST.COM (short username kudu)
I20260808 18:32:16.179944 8273 webserver.cc:460] Webserver started at http://127.7.164.125:33541/ using document root <none> and password file <none>
I20260808 18:32:16.180521 8273 fs_manager.cc:359] Metadata directory not provided
I20260808 18:32:16.180615 8273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:32:16.180853 8273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:32:16.182956 8273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/data/instance:
uuid: "5e177387671b4eb385e6d6165a2e0ad0"
format_stamp: "Formatted at 2026-08-08 18:32:16 on dist-test-slave-jztv"
server_key: "ecf1754073ac2cd535a9a4868c00d328"
server_key_iv: "e3229564a88e7cb52a45f3f94178f469"
server_key_version: "encryptionkey@0"
I20260808 18:32:16.183534 8273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/wal/instance:
uuid: "5e177387671b4eb385e6d6165a2e0ad0"
format_stamp: "Formatted at 2026-08-08 18:32:16 on dist-test-slave-jztv"
server_key: "ecf1754073ac2cd535a9a4868c00d328"
server_key_iv: "e3229564a88e7cb52a45f3f94178f469"
server_key_version: "encryptionkey@0"
I20260808 18:32:16.187055 8273 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:32:16.189409 8293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:32:16.190610 8273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:32:16.190749 8273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/wal
uuid: "5e177387671b4eb385e6d6165a2e0ad0"
format_stamp: "Formatted at 2026-08-08 18:32:16 on dist-test-slave-jztv"
server_key: "ecf1754073ac2cd535a9a4868c00d328"
server_key_iv: "e3229564a88e7cb52a45f3f94178f469"
server_key_version: "encryptionkey@0"
I20260808 18:32:16.190858 8273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:32:16.212055 8273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:32:16.214808 8273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:32:16.215015 8273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:32:16.221908 8273 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.125:39649
I20260808 18:32:16.222205 8367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.125:39649 every 8 connection(s)
I20260808 18:32:16.222975 8273 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/data/info.pb
I20260808 18:32:16.224470 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 8273
I20260808 18:32:16.224635 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-1/wal/instance
I20260808 18:32:16.224972 7825 external_mini_cluster.cc:1428] Setting key c6db5f6a598606ff1f838eaca62af902
I20260808 18:32:16.226205 8370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:32:16.230105 8370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.164.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.164.124@KRBTEST.COM" created.
Aug 08 18:32:16 dist-test-slave-jztv krb5kdc[8143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213936, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.125@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:32:16.240459 8266 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } attempt: 3
I20260808 18:32:16.244619 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.232123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58334 (local address 127.7.164.126:33651)
0808 18:32:16.232263 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.232266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.233248 (+ 982us) server_negotiation.cc:407] Connection header received
0808 18:32:16.234234 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.234239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.234302 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.234408 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.235609 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.236250 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.237463 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.237701 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.237818 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.240617 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.240624 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.240630 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.240667 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.242452 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.243063 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.243067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.243069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.243158 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.243500 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.243503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.243505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.243693 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.243801 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.244318 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.244444 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:32:16.245513 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.232603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33651 (local address 127.0.0.1:58334)
0808 18:32:16.233100 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.233141 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:16.234007 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:16.234581 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:16.234588 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:16.235003 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:16.235457 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.235470 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.236382 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.236390 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.237336 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.237346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.237857 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.237861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.238029 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.238664 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:16.238682 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:16.240393 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:16.242591 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.242599 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.242617 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.242965 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.243250 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.243253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.243255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.243395 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.243800 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:16.243808 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:16.244063 (+ 255us) client_negotiation.cc:769] Sending connection context
0808 18:32:16.244246 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 18:32:16.244564 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":148,"threads_started":1}
I20260808 18:32:16.247349 8370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:32:16 dist-test-slave-jztv krb5kdc[8143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213935, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for kudu/127.7.164.125@KRBTEST.COM
Aug 08 18:32:16 dist-test-slave-jztv krb5kdc[8143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213936, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.125@KRBTEST.COM for kudu/127.7.164.125@KRBTEST.COM
I20260808 18:32:16.253593 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.241261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:39649 (local address 127.0.0.1:51956)
0808 18:32:16.241474 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.241489 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:16.241860 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:16.242446 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:16.242449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:16.242627 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:16.242760 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.242767 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.244317 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.244321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.245252 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.245265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.246185 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.246191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.246359 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.246905 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:16.246918 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:16.249484 (+ 2566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:16.251746 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.251748 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.251750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.251953 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.252388 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.252392 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.252395 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.252480 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.253060 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:16.253063 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:16.253173 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 18:32:16.253290 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 18:32:16.253450 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260808 18:32:16.254398 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.241071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51956 (local address 127.7.164.125:39649)
0808 18:32:16.241615 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.241629 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.241660 (+ 31us) server_negotiation.cc:407] Connection header received
0808 18:32:16.242009 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.242019 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.242269 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.242432 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.242889 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.244164 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.245437 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.246061 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.246172 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.249598 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.249608 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.249621 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.249644 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.251649 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.252147 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.252154 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.252159 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.252267 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.252615 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.252623 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.252625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.252945 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.253092 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.253923 (+ 831us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.254121 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":172,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.164.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.124.keytab.
Entry for principal kudu/127.7.164.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.124.keytab.
I20260808 18:32:16.255576 7825 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhTizW/build/debug/bin/kudu
/tmp/dist-test-tasklhTizW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.124:44965
--webserver_interface=127.7.164.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.124.keytab
--principal=kudu/127.7.164.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.164.84:37205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.164.126:33651,127.7.164.125:39649,127.7.164.124:44965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:32:16.256225 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.247851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:39649 (local address 127.0.0.1:51970)
0808 18:32:16.247936 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.247950 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:16.248028 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:16.248428 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:16.248430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:16.248565 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:16.248650 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.248655 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.249485 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.249488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.250346 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.250357 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.250983 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.250987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.251090 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.251440 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:16.251459 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:16.252791 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:16.254823 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.254829 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.254831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.255014 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.255291 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.255294 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.255296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.255380 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.255801 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:16.255804 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:16.255878 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 18:32:16.255974 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 18:32:16.256091 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 18:32:16.256691 8266 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:32:16.256889 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.247914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51970 (local address 127.7.164.125:39649)
0808 18:32:16.248214 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.248221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.248232 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:32:16.248260 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.248262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.248293 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.248397 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.248758 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.249369 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.250488 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.250865 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.250949 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.252919 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.252923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.252925 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.252947 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.254719 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.255110 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.255115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.255117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.255196 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.255477 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.255479 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.255481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.255681 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.255770 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.256067 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.256205 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":139,"threads_started":1}
I20260808 18:32:16.257576 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.257240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40230)
0808 18:32:16.257401 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.257476 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 18:32:16.258405 8370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:32:16.258518 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.258190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40232)
0808 18:32:16.258395 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.258450 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260808 18:32:16.258678 8266 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:44965: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
I20260808 18:32:16.259140 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.258819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40236)
0808 18:32:16.258972 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.259059 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 18:32:16.259377 8294 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
I20260808 18:32:16.260316 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.260033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40250)
0808 18:32:16.260173 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.260235 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 18:32:16.260505 8370 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:44965: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
I20260808 18:32:16.289852 8266 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } attempt: 1
I20260808 18:32:16.290781 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.290448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40252)
0808 18:32:16.290616 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.290687 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 18:32:16.291500 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.291268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40264)
0808 18:32:16.291380 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.291433 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 18:32:16.291643 8266 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:44965: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
I20260808 18:32:16.309793 8370 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } attempt: 1
I20260808 18:32:16.310694 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.310360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40276)
0808 18:32:16.310518 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.310597 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 18:32:16.311460 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.311204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40280)
0808 18:32:16.311337 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.311397 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 18:32:16.311600 8370 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:44965: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
I20260808 18:32:16.366808 8266 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } attempt: 2
I20260808 18:32:16.367745 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.367391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40292)
0808 18:32:16.367569 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.367645 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 18:32:16.368546 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.368303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40300)
0808 18:32:16.368422 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.368476 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 18:32:16.368690 8266 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:44965: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
W20260808 18:32:16.374675 8381 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:32:16.374927 8381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:32:16.374951 8381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:32:16.374996 8381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:32:16.379143 8381 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:32:16.379194 8381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:32:16.379213 8381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:32:16.379230 8381 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:32:16.379245 8381 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:32:16.379758 8370 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } attempt: 2
I20260808 18:32:16.380517 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.380209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40316)
0808 18:32:16.380361 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.380426 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 18:32:16.381242 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.381023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40328)
0808 18:32:16.381132 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.381184 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 18:32:16.381412 8370 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.164.124:44965: Network error: Client connection negotiation failed: client connection to 127.7.164.124:44965: connect: Connection refused (error 111)
I20260808 18:32:16.384981 8381 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.164.84:37205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.164.126:33651,127.7.164.125:39649,127.7.164.124:44965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/krb5kdc/kudu_127.7.164.124.keytab
--principal=kudu/127.7.164.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.164.124:44965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.164.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:32:16.386044 8381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:32:16.387108 8381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:32:16.395773 8393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:16.395757 8394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:32:16.395761 8399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:32:16.396142 8381 server_base.cc:1034] running on GCE node
I20260808 18:32:16.396826 8381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:32:16.397936 8381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:32:16.399127 8381 hybrid_clock.cc:648] HybridClock initialized: now 1786213936399082 us; error 56 us; skew 500 ppm
Aug 08 18:32:16 dist-test-slave-jztv krb5kdc[8143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213936, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:32:16.401865 8381 init.cc:377] Logged in from keytab as kudu/127.7.164.124@KRBTEST.COM (short username kudu)
I20260808 18:32:16.402835 8381 webserver.cc:460] Webserver started at http://127.7.164.124:35499/ using document root <none> and password file <none>
I20260808 18:32:16.403359 8381 fs_manager.cc:359] Metadata directory not provided
I20260808 18:32:16.403416 8381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:32:16.403582 8381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:32:16.405552 8381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/data/instance:
uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d"
format_stamp: "Formatted at 2026-08-08 18:32:16 on dist-test-slave-jztv"
server_key: "976f95808076726e7f94e717c61420ba"
server_key_iv: "6aff151fe64d371b89048a185198a328"
server_key_version: "encryptionkey@0"
I20260808 18:32:16.406069 8381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/wal/instance:
uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d"
format_stamp: "Formatted at 2026-08-08 18:32:16 on dist-test-slave-jztv"
server_key: "976f95808076726e7f94e717c61420ba"
server_key_iv: "6aff151fe64d371b89048a185198a328"
server_key_version: "encryptionkey@0"
I20260808 18:32:16.409802 8381 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:32:16.412262 8413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:32:16.413519 8381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:32:16.413625 8381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/data,/tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/wal
uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d"
format_stamp: "Formatted at 2026-08-08 18:32:16 on dist-test-slave-jztv"
server_key: "976f95808076726e7f94e717c61420ba"
server_key_iv: "6aff151fe64d371b89048a185198a328"
server_key_version: "encryptionkey@0"
I20260808 18:32:16.413709 8381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:32:16.427623 8381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:32:16.430644 8381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:32:16.430805 8381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:32:16.437983 8381 rpc_server.cc:307] RPC server started. Bound to: 127.7.164.124:44965
I20260808 18:32:16.438352 8485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.164.124:44965 every 8 connection(s)
I20260808 18:32:16.439041 8381 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/data/info.pb
I20260808 18:32:16.441805 8486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:32:16.443564 7825 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhTizW/build/debug/bin/kudu as pid 8381
I20260808 18:32:16.443760 7825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhTizW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213640155572-7825-0/minicluster-data/master-2/wal/instance
I20260808 18:32:16.444078 7825 external_mini_cluster.cc:1428] Setting key bd45bfaaaa5c584455becd3dec3e0a90
I20260808 18:32:16.445989 8486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:32:16 dist-test-slave-jztv krb5kdc[8143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213936, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.124@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
Aug 08 18:32:16 dist-test-slave-jztv krb5kdc[8143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.124@KRBTEST.COM
I20260808 18:32:16.458812 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.447778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58340 (local address 127.7.164.126:33651)
0808 18:32:16.447934 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.447941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.448404 (+ 463us) server_negotiation.cc:407] Connection header received
0808 18:32:16.448544 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.448547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.448615 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.448707 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.449418 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.450070 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.451417 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.451759 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.451862 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.454438 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.454445 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.454449 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.454486 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.456181 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.456981 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.456984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.456985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.457105 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.457454 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.457459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.457461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.457744 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.457853 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.458479 (+ 626us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.458622 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 18:32:16.459791 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.447681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.126:33651 (local address 127.0.0.1:58340)
0808 18:32:16.448229 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.448286 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:16.448427 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:16.448762 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:16.448766 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:16.449111 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:16.449266 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.449276 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.450207 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.450243 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.451278 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.451288 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.451931 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.451938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.452097 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.452521 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:16.452545 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:16.454280 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:16.456296 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.456304 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.456316 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.456818 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.457198 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.457200 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.457201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.457332 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.457859 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:16.457864 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:16.458174 (+ 310us) client_negotiation.cc:769] Sending connection context
0808 18:32:16.458399 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 18:32:16.458736 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":1,"thread_start_us":130,"threads_started":1}
I20260808 18:32:16.459797 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.445417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40336 (local address 127.7.164.124:44965)
0808 18:32:16.445838 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.445854 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.445890 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:32:16.446823 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.446852 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.447194 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.447553 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.448560 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.449772 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.451003 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.451583 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.451740 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.454960 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.454965 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.454976 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.455006 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.456826 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.457605 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.457610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.457611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.457678 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.458005 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.458008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.458009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.458297 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.458484 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.458760 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.458880 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":264,"spinlock_wait_cycles":4992,"thread_start_us":135,"threads_started":1}
I20260808 18:32:16.461054 8486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:32:16 dist-test-slave-jztv krb5kdc[8143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213936, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.124@KRBTEST.COM for kudu/127.7.164.125@KRBTEST.COM
I20260808 18:32:16.467854 8266 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } attempt: 3
Aug 08 18:32:16 dist-test-slave-jztv krb5kdc[8143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.126@KRBTEST.COM
I20260808 18:32:16.470031 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.461565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.125:39649 (local address 127.0.0.1:51978)
0808 18:32:16.461694 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.461706 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:16.461800 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:16.462081 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:16.462084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:16.462259 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:16.462386 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.462395 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.463298 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.463301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.463820 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.463828 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.464399 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.464402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.464535 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.465035 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:16.465046 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:16.466386 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:16.468307 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.468310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.468312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.468559 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.468966 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.468968 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.468970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.469091 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.469555 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:16.469561 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:16.469673 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 18:32:16.469772 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 18:32:16.469871 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 18:32:16.470186 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.461729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51978 (local address 127.7.164.125:39649)
0808 18:32:16.461862 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.461869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.461886 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:32:16.461936 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.461939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.461983 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.462049 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.462538 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.463198 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.463939 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.464286 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.464365 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.466546 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.466551 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.466554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.466585 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.468160 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.468736 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.468740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.468742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.468836 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.469205 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.469210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.469212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.469439 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.469553 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.469856 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.469996 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:32:16.471828 8486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:32:16.472692 8370 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } attempt: 3
Aug 08 18:32:16 dist-test-slave-jztv krb5kdc[8143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213935, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.126@KRBTEST.COM for kudu/127.7.164.124@KRBTEST.COM
I20260808 18:32:16.476187 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.462662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58342 (local address 127.7.164.126:33651)
0808 18:32:16.463350 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.463353 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.463362 (+ 9us) server_negotiation.cc:407] Connection header received
0808 18:32:16.463390 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.463392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.463447 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.463535 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.464206 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.464861 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.465767 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.465967 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.466048 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.468608 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.468613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.468616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.468640 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.470921 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.474346 (+ 3425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.474350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.474352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.474431 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.474775 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.474779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.474782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.475018 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.475825 (+ 807us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.475913 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.476028 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620}
Aug 08 18:32:16 dist-test-slave-jztv krb5kdc[8143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213936, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.124@KRBTEST.COM for kudu/127.7.164.124@KRBTEST.COM
I20260808 18:32:16.478734 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.468269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40346)
0808 18:32:16.468443 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.468457 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:16.468552 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:16.468797 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:16.468800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:16.468988 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:16.469119 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.469127 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.470048 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.470052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.470921 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.470928 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.471560 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.471564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.471679 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.472123 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:16.472136 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:16.474243 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:16.476288 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.476291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.476293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.476560 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.476852 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.476855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.476857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.476974 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.477415 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:16.477420 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:16.478257 (+ 837us) client_negotiation.cc:769] Sending connection context
0808 18:32:16.478388 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 18:32:16.478556 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260808 18:32:16.478749 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.468351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40346 (local address 127.7.164.124:44965)
0808 18:32:16.468468 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.468472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.468514 (+ 42us) server_negotiation.cc:407] Connection header received
0808 18:32:16.468647 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.468652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.468702 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.468773 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.469259 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.469918 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.471082 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.471436 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.471527 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.474361 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.474365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.474367 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.474390 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.476048 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.476662 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.476665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.476667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.476746 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.477078 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.477081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.477083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.477301 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.477401 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.478472 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.478587 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 18:32:16 dist-test-slave-jztv krb5kdc[8143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213936, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.164.125@KRBTEST.COM for kudu/127.7.164.124@KRBTEST.COM
I20260808 18:32:16.482715 8266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7: Bootstrap starting.
I20260808 18:32:16.483660 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.473118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40360)
0808 18:32:16.473676 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.473691 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:16.473792 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:16.474043 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:16.474046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:16.474266 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:16.474366 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.474373 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.475488 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.475493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.476312 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.476323 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.476981 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.476985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.477085 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.477507 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:16.477523 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:16.479644 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:16.481804 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.481810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.481815 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.482110 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.482485 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.482489 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.482491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.482594 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.483102 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:16.483105 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:16.483213 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 18:32:16.483307 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 18:32:16.483464 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477}
Aug 08 18:32:16 dist-test-slave-jztv krb5kdc[8143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.164.125@KRBTEST.COM
I20260808 18:32:16.484409 8492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.473334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40360 (local address 127.7.164.124:44965)
0808 18:32:16.473667 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.473674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.473739 (+ 65us) server_negotiation.cc:407] Connection header received
0808 18:32:16.473880 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.473882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.473912 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.474022 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.474514 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.475339 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.476473 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.476848 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.477267 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.479789 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.479792 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.479795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.479820 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.481680 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.482256 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.482259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.482261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.482350 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.482727 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.482731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.482733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.482980 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.483928 (+ 948us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.484009 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.484175 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":268,"spinlock_wait_cycles":3200,"thread_start_us":136,"threads_started":1}
I20260808 18:32:16.484651 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.472415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.164.124:44965 (local address 127.0.0.1:40358)
0808 18:32:16.472509 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 18:32:16.472525 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:32:16.472597 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:32:16.473079 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:32:16.473082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:32:16.473250 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:32:16.473350 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.473356 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.475427 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.475431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.476150 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:32:16.476161 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.476840 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.476844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:32:16.476910 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.477536 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:32:16.477553 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:32:16.480076 (+ 2523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:32:16.482801 (+ 2725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.482803 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.482805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.483048 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.483314 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:32:16.483316 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:32:16.483318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:32:16.483391 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:32:16.483785 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:32:16.483787 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:32:16.483858 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 18:32:16.483951 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 18:32:16.484102 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"spinlock_wait_cycles":3840}
I20260808 18:32:16.485044 8491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.472480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40358 (local address 127.7.164.124:44965)
0808 18:32:16.472845 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.472854 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.472866 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:32:16.472896 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.472899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.472945 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.473039 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.473491 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.475073 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.476284 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.476644 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.476740 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.480473 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.480477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.480479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.480503 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.482681 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.483151 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.483154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.483156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.483230 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.483493 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.483495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.483497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.483671 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.483754 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.484277 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.484385 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":164,"threads_started":1}
I20260808 18:32:16.487803 8486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d: Bootstrap starting.
I20260808 18:32:16.487838 8370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0: Bootstrap starting.
I20260808 18:32:16.488085 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.477770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51988 (local address 127.7.164.125:39649)
0808 18:32:16.477873 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.477877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.477890 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:32:16.477935 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.477937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.477986 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.478077 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.478895 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.479595 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.480706 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.481111 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.481221 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.484381 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.484386 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.484392 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.484418 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.486183 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.486810 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.486815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.486817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.486914 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.487231 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.487234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.487236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.487436 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.487609 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.487834 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.487956 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:32:16.490468 8486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d: Neither blocks nor log segments found. Creating new log.
I20260808 18:32:16.490505 8370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0: Neither blocks nor log segments found. Creating new log.
I20260808 18:32:16.491297 8486 log.cc:824] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d: Log is configured to *not* fsync() on all Append() calls
I20260808 18:32:16.491326 8370 log.cc:824] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0: Log is configured to *not* fsync() on all Append() calls
I20260808 18:32:16.491792 7825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:32:16.492486 8266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7: Neither blocks nor log segments found. Creating new log.
I20260808 18:32:16.493321 8266 log.cc:824] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7: Log is configured to *not* fsync() on all Append() calls
I20260808 18:32:16.493595 8370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0: No bootstrap required, opened a new log
I20260808 18:32:16.493048 8231 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0278df41af8549499b0b1b50dbd476b7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58342
I20260808 18:32:16.495477 8486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d: No bootstrap required, opened a new log
I20260808 18:32:16.495700 8266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7: No bootstrap required, opened a new log
I20260808 18:32:16.497726 8370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } }
I20260808 18:32:16.497956 8370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e177387671b4eb385e6d6165a2e0ad0, State: Initialized, Role: FOLLOWER
I20260808 18:32:16.498296 8370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } }
I20260808 18:32:16.498371 8266 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } }
I20260808 18:32:16.498587 8266 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0278df41af8549499b0b1b50dbd476b7, State: Initialized, Role: FOLLOWER
I20260808 18:32:16.498895 8266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } }
I20260808 18:32:16.499523 8500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } } }
I20260808 18:32:16.499716 8370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:32:16.499847 8500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:32:16.500221 8266 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:32:16.500099 8486 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } }
I20260808 18:32:16.500275 8231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 18:32:16.500321 8486 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d2cdf4d2e240d38aacd1b117c47f8d, State: Initialized, Role: FOLLOWER
I20260808 18:32:16.500383 8231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:32:16.500612 8486 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } }
I20260808 18:32:16.501794 8231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } }
I20260808 18:32:16.501863 8486 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:32:16.502857 8231 leader_election.cc:288] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [CANDIDATE]: Term 1 election: Requested vote from peers 5e177387671b4eb385e6d6165a2e0ad0 (127.7.164.125:39649), b4d2cdf4d2e240d38aacd1b117c47f8d (127.7.164.124:44965)
I20260808 18:32:16.504396 8447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0278df41af8549499b0b1b50dbd476b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d"
I20260808 18:32:16.505146 8447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:32:16.506626 8447 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0278df41af8549499b0b1b50dbd476b7 in term 1.
I20260808 18:32:16.504516 8330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0278df41af8549499b0b1b50dbd476b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5e177387671b4eb385e6d6165a2e0ad0"
I20260808 18:32:16.506801 8330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0 [term 0 FOLLOWER]: Advancing to term 1
W20260808 18:32:16.508050 8531 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:32:16.508117 8531 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:32:16.508208 8330 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0278df41af8549499b0b1b50dbd476b7 in term 1.
I20260808 18:32:16.508452 8507 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } } }
I20260808 18:32:16.508582 8507 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:32:16.509008 8185 leader_election.cc:302] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0278df41af8549499b0b1b50dbd476b7, b4d2cdf4d2e240d38aacd1b117c47f8d; no voters:
W20260808 18:32:16.511931 8551 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:32:16.512025 8551 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:32:16.513832 8500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:32:16.514160 8500 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [term 1 LEADER]: Becoming Leader. State: Replica: 0278df41af8549499b0b1b50dbd476b7, State: Running, Role: LEADER
I20260808 18:32:16.514593 8273 master_runner.cc:263] No leader master found from master 5e177387671b4eb385e6d6165a2e0ad0
I20260808 18:32:16.514673 8273 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:32:16.514999 8161 master_runner.cc:263] No leader master found from master 0278df41af8549499b0b1b50dbd476b7
I20260808 18:32:16.515074 8161 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:32:16.515318 8500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } }
I20260808 18:32:16.516743 8381 master_runner.cc:263] No leader master found from master b4d2cdf4d2e240d38aacd1b117c47f8d
I20260808 18:32:16.516803 8381 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:32:16.517544 8500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0278df41af8549499b0b1b50dbd476b7. Latest consensus state: current_term: 1 leader_uuid: "0278df41af8549499b0b1b50dbd476b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } } }
I20260808 18:32:16.517656 8500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [sys.catalog]: This master's current role is: LEADER
W20260808 18:32:16.505756 8533 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:32:16.517874 8533 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:32:16.506016 8499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } } }
I20260808 18:32:16.517979 8499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:32:16.519366 8568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:32:16.520151 8568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:32:16.525575 8446 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d [term 1 FOLLOWER]: Refusing update from remote peer 0278df41af8549499b0b1b50dbd476b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:32:16.526389 8500 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:32:16.527128 8330 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0 [term 1 FOLLOWER]: Refusing update from remote peer 0278df41af8549499b0b1b50dbd476b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:32:16.527872 8564 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:32:16.530771 8507 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0278df41af8549499b0b1b50dbd476b7. Latest consensus state: current_term: 1 leader_uuid: "0278df41af8549499b0b1b50dbd476b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } } }
I20260808 18:32:16.530922 8507 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:32:16.532081 8499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0278df41af8549499b0b1b50dbd476b7. Latest consensus state: current_term: 1 leader_uuid: "0278df41af8549499b0b1b50dbd476b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } } }
I20260808 18:32:16.532228 8499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:32:16.542416 8500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0278df41af8549499b0b1b50dbd476b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } } }
I20260808 18:32:16.542570 8500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:16.542605 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.522131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.7.164.126:33651)
0808 18:32:16.522276 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.522281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.524583 (+ 2302us) server_negotiation.cc:407] Connection header received
0808 18:32:16.524759 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.524765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.524818 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.524868 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.527409 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.528167 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.531161 (+ 2994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.531512 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.531613 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.535022 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.535026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.535029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.535060 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.536884 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.538691 (+ 1807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.538698 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.538701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.538800 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.539141 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.539145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.539146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.539408 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.539526 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.542307 (+ 2781us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.542453 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:32:16.543136 8507 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0278df41af8549499b0b1b50dbd476b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } } }
I20260808 18:32:16.543264 8507 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:32:16.543870 8581 mvcc.cc:205] Tried to move back new op lower bound from 7316332284000047104 to 7316332283971772416. Current Snapshot: MvccSnapshot[applied={T|T < 7316332284000047104}]
I20260808 18:32:16.544144 8576 mvcc.cc:205] Tried to move back new op lower bound from 7316332284000047104 to 7316332283971772416. Current Snapshot: MvccSnapshot[applied={T|T < 7316332284000047104}]
I20260808 18:32:16.544610 8564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0278df41af8549499b0b1b50dbd476b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } } }
I20260808 18:32:16.544741 8564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7 [sys.catalog]: This master's current role is: LEADER
I20260808 18:32:16.546241 8499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0278df41af8549499b0b1b50dbd476b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0278df41af8549499b0b1b50dbd476b7" member_type: VOTER last_known_addr { host: "127.7.164.126" port: 33651 } } peers { permanent_uuid: "5e177387671b4eb385e6d6165a2e0ad0" member_type: VOTER last_known_addr { host: "127.7.164.125" port: 39649 } } peers { permanent_uuid: "b4d2cdf4d2e240d38aacd1b117c47f8d" member_type: VOTER last_known_addr { host: "127.7.164.124" port: 44965 } } }
I20260808 18:32:16.546376 8499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:32:16.547164 8491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.522905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40370 (local address 127.7.164.124:44965)
0808 18:32:16.523013 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.523017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.525988 (+ 2971us) server_negotiation.cc:407] Connection header received
0808 18:32:16.526197 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.526201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.526285 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.526349 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.529551 (+ 3202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.530311 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.534178 (+ 3867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.534571 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.534645 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.539886 (+ 5241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.539890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.539893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.539924 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.541678 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.543165 (+ 1487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.543169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.543170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.543272 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.543629 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.543635 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.543637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.543881 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.544003 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.546839 (+ 2836us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.546980 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:32:16.547163 8585 mvcc.cc:205] Tried to move back new op lower bound from 7316332284000047104 to 7316332283971772416. Current Snapshot: MvccSnapshot[applied={T|T < 7316332284000047104}]
I20260808 18:32:16.549060 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:16.518519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51998 (local address 127.7.164.125:39649)
0808 18:32:16.518643 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.518649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.523533 (+ 4884us) server_negotiation.cc:407] Connection header received
0808 18:32:16.523738 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.523742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.523796 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.523850 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.532462 (+ 8612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.533204 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.534373 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:16.534696 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:16.534771 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:16.544415 (+ 9644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:32:16.544420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:32:16.544424 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:32:16.544451 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:32:16.546272 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.547632 (+ 1360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.547638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.547643 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.547740 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:32:16.548069 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:32:16.548072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:32:16.548074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:32:16.548296 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:32:16.548416 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:16.548760 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 18:32:16.548881 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:32:16.555697 8568 catalog_manager.cc:1347] Generated new cluster ID: 5e8bdcbc888649f9bbc87b2f54911665
I20260808 18:32:16.555776 8568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:32:16.599263 8568 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:32:16.600607 8568 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:32:16.624895 8568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0278df41af8549499b0b1b50dbd476b7: Generated new TSK 0
I20260808 18:32:16.625835 8568 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:32:16.664831 8380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:16.661390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52006 (local address 127.7.164.125:39649)
0808 18:32:16.661534 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.661541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.663674 (+ 2133us) server_negotiation.cc:407] Connection header received
0808 18:32:16.663882 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.663886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.663943 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.663998 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.664693 (+ 695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52006: BlockingRecv error: recv got EOF from 127.0.0.1:52006 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 18:32:16.666322 8269 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:16.662297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58348 (local address 127.7.164.126:33651)
0808 18:32:16.662435 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.662439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.665347 (+ 2908us) server_negotiation.cc:407] Connection header received
0808 18:32:16.665545 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.665549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.665604 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.665657 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.666152 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58348: BlockingRecv error: recv got EOF from 127.0.0.1:58348 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 18:32:16.668010 8491 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:16.663050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40380 (local address 127.7.164.124:44965)
0808 18:32:16.663172 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.663176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.666873 (+ 3697us) server_negotiation.cc:407] Connection header received
0808 18:32:16.667135 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.667141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.667203 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.667260 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.667860 (+ 600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40380: BlockingRecv error: recv got EOF from 127.0.0.1:40380 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 18:32:16.672776 8380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:16.671182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52016 (local address 127.7.164.125:39649)
0808 18:32:16.671288 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.671293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.671790 (+ 497us) server_negotiation.cc:407] Connection header received
0808 18:32:16.672004 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.672008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.672090 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.672142 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.672665 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52016: BlockingRecv error: recv got EOF from 127.0.0.1:52016 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 18:32:16.678198 8380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:16.676486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52032 (local address 127.7.164.125:39649)
0808 18:32:16.676589 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.676593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.677151 (+ 558us) server_negotiation.cc:407] Connection header received
0808 18:32:16.677334 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.677338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.677457 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.677513 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.678074 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52032: BlockingRecv error: recv got EOF from 127.0.0.1:52032 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 18:32:16.683866 8380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:16.682284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52044 (local address 127.7.164.125:39649)
0808 18:32:16.682409 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.682414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.682951 (+ 537us) server_negotiation.cc:407] Connection header received
0808 18:32:16.683138 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.683141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.683192 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.683243 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.683750 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52044: BlockingRecv error: recv got EOF from 127.0.0.1:52044 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 18:32:16.691948 8380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:16.690260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52056 (local address 127.7.164.125:39649)
0808 18:32:16.690417 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.690422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.690961 (+ 539us) server_negotiation.cc:407] Connection header received
0808 18:32:16.691175 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.691179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.691234 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.691290 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.691843 (+ 553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52056: BlockingRecv error: recv got EOF from 127.0.0.1:52056 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 18:32:16.704375 8380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:16.702640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52060 (local address 127.7.164.125:39649)
0808 18:32:16.702769 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.702773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.703386 (+ 613us) server_negotiation.cc:407] Connection header received
0808 18:32:16.703594 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.703598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.703652 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.703701 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.704245 (+ 544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52060: BlockingRecv error: recv got EOF from 127.0.0.1:52060 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 18:32:16.724864 8380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:16.723007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52064 (local address 127.7.164.125:39649)
0808 18:32:16.723143 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.723150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.723804 (+ 654us) server_negotiation.cc:407] Connection header received
0808 18:32:16.724025 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.724029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.724086 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.724139 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.724739 (+ 600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52064: BlockingRecv error: recv got EOF from 127.0.0.1:52064 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 18:32:16.760492 8380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:16.759382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52070 (local address 127.7.164.125:39649)
0808 18:32:16.759566 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.759571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.759658 (+ 87us) server_negotiation.cc:407] Connection header received
0808 18:32:16.759794 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.759796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.759847 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.759932 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.760390 (+ 458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52070: BlockingRecv error: recv got EOF from 127.0.0.1:52070 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260808 18:32:16.828342 8380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:16.826971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52076 (local address 127.7.164.125:39649)
0808 18:32:16.827165 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.827170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.827326 (+ 156us) server_negotiation.cc:407] Connection header received
0808 18:32:16.827547 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.827550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.827605 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.827707 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.828188 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52076: BlockingRecv error: recv got EOF from 127.0.0.1:52076 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260808 18:32:16.960103 8380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:16.958680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52084 (local address 127.7.164.125:39649)
0808 18:32:16.958822 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:16.958827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:16.959020 (+ 193us) server_negotiation.cc:407] Connection header received
0808 18:32:16.959192 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:16.959198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:16.959254 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:16.959367 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:16.959878 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52084: BlockingRecv error: recv got EOF from 127.0.0.1:52084 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 18:32:17.219612 8380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:32:17.218500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52100 (local address 127.7.164.125:39649)
0808 18:32:17.218640 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:17.218645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:17.218771 (+ 126us) server_negotiation.cc:407] Connection header received
0808 18:32:17.218896 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:17.218899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:17.218949 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:17.219037 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:32:17.219480 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52100: BlockingRecv error: recv got EOF from 127.0.0.1:52100 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:32:17.513389 8551 catalog_manager.cc:1259] Loaded cluster ID: 5e8bdcbc888649f9bbc87b2f54911665
I20260808 18:32:17.513522 8551 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d: loading cluster ID for follower catalog manager: success
I20260808 18:32:17.515354 8551 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d: acquiring CA information for follower catalog manager: success
I20260808 18:32:17.516642 8551 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b4d2cdf4d2e240d38aacd1b117c47f8d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:32:17.519027 8533 catalog_manager.cc:1259] Loaded cluster ID: 5e8bdcbc888649f9bbc87b2f54911665
I20260808 18:32:17.519093 8533 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0: loading cluster ID for follower catalog manager: success
I20260808 18:32:17.520638 8533 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0: acquiring CA information for follower catalog manager: success
I20260808 18:32:17.521667 8533 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5e177387671b4eb385e6d6165a2e0ad0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:32:17.737648 8686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:17.733715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52114 (local address 127.7.164.125:39649)
0808 18:32:17.733930 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:17.733934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:17.733949 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:32:17.734211 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:17.734218 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:17.734395 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:17.734508 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:32:17.734809 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:17.735468 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:17.736327 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:17.736545 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:17.736620 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:17.736899 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:32:17.737325 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:32:17.737419 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:17.737473 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 18:32:17.737523 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":67,"threads_started":1}
I20260808 18:32:17.744413 8696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:17.740812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40394 (local address 127.7.164.124:44965)
0808 18:32:17.741049 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:17.741053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:17.741065 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:32:17.741142 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:17.741144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:17.741252 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:17.741348 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:32:17.741620 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:17.742128 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:17.743020 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:17.743244 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:17.743322 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:17.743624 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:32:17.744013 (+ 389us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:32:17.744111 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:17.744172 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 18:32:17.744218 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":89,"threads_started":1}
I20260808 18:32:17.751478 8686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:17.747612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52126 (local address 127.7.164.125:39649)
0808 18:32:17.747728 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:17.747733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:17.747892 (+ 159us) server_negotiation.cc:407] Connection header received
0808 18:32:17.748102 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:17.748106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:17.748158 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:17.748206 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:32:17.748660 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:17.749393 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:17.750271 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:17.750600 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:17.750708 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:17.750972 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:32:17.751052 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:32:17.751125 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:17.751246 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 18:32:17.751300 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 18:32:17.752002 8696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:17.747902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40408 (local address 127.7.164.124:44965)
0808 18:32:17.748040 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:17.748046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:17.748060 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:32:17.748107 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:17.748109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:17.748158 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:17.748230 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:32:17.748488 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:17.749268 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:17.750550 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:17.750838 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:17.750948 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:17.751541 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:32:17.751629 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:32:17.751707 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:17.751809 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 18:32:17.751858 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:32:17.753226 8709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:32:17.747495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.7.164.126:33651)
0808 18:32:17.748030 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0808 18:32:17.748035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:32:17.748049 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:32:17.748145 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:32:17.748153 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:32:17.748319 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:32:17.748448 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:32:17.749543 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:17.750354 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:17.751496 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:32:17.752050 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:32:17.752204 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:32:17.752539 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:32:17.752850 (+ 311us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:32:17.752922 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 18:32:17.752983 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 18:32:17.753033 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":86,"threads_started":1}
I20260808 18:32:17.756639 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 8161
I20260808 18:32:17.769057 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 8273
I20260808 18:32:17.782152 7825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhTizW/build/debug/bin/kudu with pid 8381
2026-08-08T18:32:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3333 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6691 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (297623 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST